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FOREWORD 



This publication is one in a series of handbooks:.on evaluation produced by the National 
Center for Research in Vocational Education. A prinnary purpose for this handbook series is to 
offer^. assistance to persons working to increase the quality of vocational education. Reflected in 
all put)lications of the handbook series is the intent to advance the theory and practice of evaluation. 
Specifically/ the material presented in this handbook will help provoke, stimulate, and lead the way 
toward more reliable and valid assessment of employer satisfaction of vocational education. 

This handbook was developerfby the Evaluation and Policy Division of the National Center for 
Research in Vocational Education under a contract with the U,S, Department of Education, Office 
of Vocational and Adult Education. The National Center is particularly indebted to Stephen J. . 
Franchak, Project Director; Elizabeth Jen, Graduate Research Associate; and Eliseo Ponce, Graduate 
Research Associate, who had the primary responsibilities for the preparation of this document. Also, 
recognition and appreciation are extended to Larry L. Smiley, Associate Professor of Education 
Administration, University of North Dakota, vyho contributed an original draft which formed a 
primary base for the development of this handbook. 

In addition, significant contributions to the development of this publication were made by " ' 
other members of the National Center's Evaluation and Policy Division, including N? L.\l\/IcCaslin, 
Associate Director; F. L. Mc^inney, Program Director; and;Lynn Brant, Graduate Research Associate. 
The National Center extends its appreciation to the,followmg,state and local education personnel, 
who reviewed the draft outline of the handbook: staff members 'from the Research Coordinating Unit, 
Alabama Department of Education; Herb Rand and Mark Headfick, Division of Vocational Education,' 
Florida Department of Education; Cheryl A. Rigby and, Aaron Gaines, Lively Area Vocational, Center,' 
Leon County, Florida; Rose Mary Bengel, Maryland.State.Department of Education; Andrea Kelly 
and Ken Lake, South Carolina Department of Education; and Steven Bishopp, Commission of 
Vocational Education, State of Washington. 

We are also grateful to the eight members of the National Center's Evaluation Technical Advisory 
Panel: George C. Copa, University of Minnesota; Toni Hall, Navarro College,^ Texas; Ruth P. Hughes, 
Iowa State University; William Morris, Chancellor's Office, California Community Colleges; Douglas 
Patterson, Alabama State Department of Education; Delores Robinson, Florida- State University; 
Robert Spillman, Kentucky State Department of Education; and to Tim L. Wentling, University'of 
Illinois. Credit is given to the following reviewers of the draft copy: Fred A. Snyder, President, 
Canadian Corporation, Texas; Ervin Geigle, Consultant, Minnesota; and Roy L. Butler, Research \ 
Specialist II, the National Center for Research in Vocational Education, The Ohio State University. 

■ ^ * ' '■ 

■ ■ 

Appreciation is extended to Mary Sue Birtlef, Jeanette McConau'ghy, and Janet Kiplinger, 
who provided editorial assistance. In addition, thanks are extended to Ernie Spaeth and his staff 
for^their efforts regarding the graphics and printing of this publication. 
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Finally, a special note of appreciation is extended to Sherry White/who had the major secre- 
tarial responsibilities for this publication; to Venita Rartimell, Kathy Haycook, Kathleen Medley 
and Priscilla Ciulla for their typing assistance. Also, thanks are extended to Marilyn Orlando for her 
secretarial assistance. 
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Executive Director. 
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EXECUTIVE SUMMARY 



- The effects of the Education Amendments of 1976 on evaluation practices are beginning to be 
realized The complex nature of evaluation, its theory and methods, continue to face those state 
and local evaluators charged with planning and conducting evaluation studiek Recent assessments 
of state and local evaluation practices indicate that formal employer follow-iip data are collected 
much less consistently than student follow-up. Reasons for this lack of attention to this type of 
evaluation activity are varied. y /r- . 

According to the literature, evaluation efforts have been considered inadequate, particularly in 
regard to having an impact upon program improvement. Several reasons have been posited to support 
this statement, such as (1 ) programs often have multiple goals that are difficult to evaluate (2) pro- 
grams are complex and dynamic, and (3) inappropriate evaluation designs or no elaluation'designs 
are used to address the evaluation goals and objectives. However, evaluation efforts ara included as an 
integral part of state and local education program, activities for program improvement and account- 
ability. There is a need to provide evaluators and users of evaluation results with ^r-Jchlinesand 
practices for their tasks and responsibilities. 

This handbook was designed to identify exemplary strategies and pro «' iii?-^; ■ fe- v^ asssng 
employer satisfaction with former vocational education studer 's' training ai~- j puru*. fiance 
Specifically, the handbook provides ideas for initiating such an assessment, or for 'inl- r-'-^ng ,jpon ■ 
current practices. Also, it focuses on problems and issues the vocational education es,^lu«cor encounters 
in doing this type of evalgatibn activity. 

The target audience for this handbook is varied. Potential users include, but are net limited to 
state and local vocational education program evaluators, personnel directly responsible for conducting 
employer follow-up studies, and members of advisory committees. 

■ The handbook is organized to serve as a "ready reference" on the subject of employer follow-up. 
It IS divided into five sections and numerous subsections, followed by selected and annotated bibliog- 
raphies, and an appendix. ° ^ 

Chapter I provides a comprehensive oyerviewTof contents. Chapter I i presents a rationale for 
conducting employer follow-up. Chapter I I I discusses problems and issues in assessing employer 
satisfaction with former vocational education students' training and jot) performance. Chapter I V 
presents an overview of the differences and commonalities between employer satisfaction with training 
and job performance. Chapter V examines the steps required in designing, planning, and implementing 
an employer follow-up. A major portion of this section Is adapted from a previous National Center 
Evaluation Handbook: Guidelines and Practices for Conducting Follow-up Studies, by Franchak and ' 
Spirer (1978). ' 

The handbook also provides a Selected Bibliography and an Annotated Bibliography containing 
detailed summaries of major studies conducted on state and local levels to assess employer satisfac- 
tion with former vocational students' training and job performance. \ 
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The Appendix includes a sample employer ques' ionnaire, which may be modified or adapted 
to meet the evaluator's particular needs in conducting an erpployer follow-up. V. ^ 

Where appropriate, chercklists are provided to encourage the reader to think through va'rious 
processes or steps in designing, planning, and conducting employer follow-up studies. Also, key 
references are incorporated into the text to provide thf reader with relevant information about 
major concepts and specific content to make informed decisions about alternative strategies and 
procedures. ^ 

The major value of the handbook is a comprehensive view of the many-faceted concepts, 
strategies, and pt oceduros associated with the assessment of employer satisfaction yvith former 
vocational education students' training and job performance. Moreover, it is intended to focus on 
the problems one encounters as a vocational education evaluator, how these.problems have been ' 
solved or not solved, assesi>meint trends, and what still rema^ins for further study. 



CHAPTER I 



INTRODUCTION 



Backgroijnd " - 

The purpose of this handbook is to identify exemplary strategies and procedures to help guide - 
the design and conduct of studies for assessing employer satisfaction with former vocationa! education 
students' training and job performance. Specifically, the handbook provides ideas fpr initiating such 
an assessment or for improving upon current practice. Also, it focuses on problems and issues related 
to this type of evaluation activity. This assessment bf employer satisfaction falls under the category 
of program evaluation or impact evaluation. 

The content for this handbook yvas developed primarily from a syn,ttiesis of concepts, practices, 
and materials from the current state of the art. Moreover, the information base was developed from 
• a survey of the literature on follow-up studies and job qerformartce, interviews with seiected^state 
and local practitioners, a review of relevant laws and regulations, and a study of exemplary evaluation 
systems and employer follow-up instruments. This publication draws extensively from-previous 
INIationa I Center evaluation projects. Specifically, information from the following publications'has 
been used in the development of this handbook: Franchak and Spirer (1978), Gray et al. (1978), 
Asche and Vogler (1980), Stevenson (1979), Adams and Walker (1979), Darcy (1979), and Farley; 
(1979). ' 

The handbook is intended to provide a connprehensive view of the many-faceted concepts, 
strategies, and procedures associated with the assessment of employer satisfaction with former 
vocational education students' training and job performance/ Moreover, it is intended to focus on 
problems one encounters as a vocational education evaluator, how these problems have been solved 
or not solved, the main trends in evaluation, and what still remains for further study. The practitioners 
doing' the employer follow-up are the experts in the processes which are the essence of this handbook. 
Not only have we asked them for specific information that is difficult to find from published sources, 
.but, even^more important, we have looked to those practitioners to provide guidance in developing 
the contents; and have asked for their review of the document for its meaningfujness and usefulness. 



Audience 

" - ' " /' • • ' ' 

The target audience for this handbook is varied. The primary audience is state and local 
vocational education evaluators. Potential users include, but are not limited to, vocational education 
advisory committee m^bers, and personnel responsible for program development. Naturally, not 
all chapters of this handbook are equally important to each member of such a varied group cf users. 
The advisory committee member has vastly different; needs from the program staff member responsible 
for the conduct of an employer folTbvy-up study. And people undertaking for the.first time the design ' 
and conduct of an employer follow-up versus people who^ after ten years of follow-up study experience, 
plan to redesign their system to satisfy federal or stats reporting requirements, are examples that V 



reflec* varied information needs. Thus, it is recommended that the initiai task of readers be to 
examine the table of contents and decide vyhich section or subsections are rjiost relevant to their 
needs. Another aid in identifying appropriate content is toVeview and complete the checklist for 
Assessing Individual Problems in the following subsection. ' 

To the extent possible, each chapter of the handbook has been written to provide readers with 
enough relevant information about major concepts and specific content to make informed decisions 
about alternative strategies and procedures. References have been incorporated into the text to assist 
further in this task. 

Organization 

a- ; • ■ • * 

The handbook is organized to serve as a ready reference on the subject of employer follow-up. 
It IS divided into five sections and numerous subsections followed by selected and annotated bibliog- 
raphies and an appendix. Each chapter has been written to provide readers with^enough relevant 
.inf prmation.about specific content to make ir formed decisions about altei-native strategies and 
procedures. References have been incorporated into the text to assist in this task. 

Chapter I provides a conriprehensive overview of contents. Chapter II presents a rationale for 
conducting enriployer follow-up. Chapter III discusses issues and problems in assessing employer 
satisfaction with former vocational education students' training and job performance. 

The appendix includes, a sample questionnaire, which may be modified or adapted to me^t the 
evaluator's particular needs in conducting an employer follow-up ^tudy. This instrument contains 
questions addressing both employer satisfaction with both the training and job performance of former 
vocational education students. 

A glossary is included to provide the f-eader mxh a definition of terms corisideredimportaht in 

employer followrup studies. ^ 

* ■ ' ■ ■ . ,- • ' , . ■• 

The handbook also provides a selected bibliography and an annotated bibliography containing 
detailed summaries of studies conducted onVa^e and^local levels'to assess employer satisfaction ' 
with training and vyith job performances of foM:q^er vocational students; ' ^ 

The following checklist is recommended as a starting point for the re^ader to identify and 
prioritize employer follow-up study problem's. 



CHECKLIST 1 
Assessing Individual Problems 



Do these problems exist for you? 


Yes 


No 


Rand Order by 
Extent of Problem 


1. Lack of understanding of the 
purpose for assessing "employer 
satisfaction 


D 


D 




2. Lack of understanding of 

appropriate assessment procedures 


D 


D 




3. Lack of clear definitions of terms 
' -used in the data collection 


D 


D . 




4. Inadequate response rates from 

cllI|JI,uyci S 


D ■ 


,n 




5. Lack of understanding concerning 
report generation for user groups 




D. 




6. Unwitjingness of.decision makers 
to use^employer satisfaction data 




D • 




7. Lack of understanding of the 

relationship between assessment ^ 
of training and job performance 
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CHAPTER II 

RATIONALE FOR ASSESSING EMPLOYER SATISFACTION 



Vocational educators at the state and local levels are under increasing pressure to plan and 
conduct meaningful and useful evaluations. Since the passage of the 1963 Vocational Education Act, 
increased effort has been expended toward determining the quality of vocational edu<;ation program's. 
For the purposes of this handbook, quality is comprised of three factors/ 

1- Effectiveness: Does the vocational ^rogram>enable students to— 

a. meet program objectives, and 

b. perform satisfactorily on the job? 

2. Efficiency:^ Doing things in the most cost;effective manner. Is there an apparent 
waste in the:vocational program of student time. Instructional time, materials and 
equiprnent^and use of facilities? , ^ - 

3. /?e/ei/air7ce; -Are tlie knowledge and skills in the program— 

a. performed on the job, 

b. critical to job success, and ' ^ 

c. consistent with on-the-job practices, .> . 

Legislators and the general public.are often frustrated in their attempts to determine how well 
• vocational education is serving the community. Complaints of poor vocational programs may be 
reported in the media (e.g., Maerof f 1 979; Raspberry 1979a) or brought directly to the attention of 
the state and local directors of vocationaj education (e.g., Illinois State and Locally Developed 
Evaluation System, Oklahoma Process-Product Evaluation. System, Ohio Pride System, California 
Copes, Texas TEX-SIS, Michigan Evaluation System, Alabama Evaluation System, and Colorado 
Evaluation System). These states provide a systematic review of programs via self-evaluation and 
third-party on-site program reviews. The results are presented to the school director, who then 
prepares a plan to address the problem as identified in the evaluation results. Positive results, likewise, 
'come to light in a similar manner. Typically, however, a comprehensive view of the quality of voca-, 
tional Education is not available in terms that meet everyone's satisfaction. 



^ ^Legislative Requirepients ' 

^ Current legislation, the Education Amendments of 1976, offers prescriptive evaluation require- 
ments-ior-th'dise 'State (SEA) and local (LEA) education agencies receiving federal funds. These 
requirements found in Sec. 112 (b) (1) are:. 

(A) each state shall, during the five-year period of.the state plan, evaluate the effectiveness 
of each program within the state being assisted with funds available under this Act; 
' and the results of tHeseievaluatiohs shall be used to revise the state's programs, and 
. shall be made re adily available to the state advisory councils; and 



(B) each state shall evaluate, by using data collected, wherever possible, by statistically 
valid sampling techniques, each such program within the state which purports to 
impart entry-level job skills according to the extent to which program completers 
- and leavers— 

i. find employment in occupations related to their training, and 
. ii. are considered by their employers to be well trained and prepared for employment, 
except that in no case can pursuit of additional education or training by program 
completers or leavers be considered negatively in these evaluations. 

This focus on preparing students for productive participation in the labor market is not new. It 
dates back to the 1917 Smith-Hughes Act, the beginning of federally funded vocational edud^tion. 
Over the years this focus, was retained in subsequent amendments to that Act. More recently, the 
congressional intent was reinforced in the Vocational Education Act of 1976: 

The Committee has specified two criteria (Sec. 112 (b) (1)-(B) (i, ii)to judge 
the effectiveness of programs because in our opinion, they show most clearly 
whether persons trained in vocational programs are showing the results of 
such training {italicsadded] . (U.S. Congress, House Report No 94-1085 
1976, p, 20. . ■ . . 

Economic. Conditions 

Education now finds itself having to compete for government funding with other agencies for 
limited human and financial resources. There seems to be a trend developing which indicates that 
decisions by policymaking bodies regarding resource allocations are being made with increasing 
frequency on the basis of evidence of program quality (effectiveness, efficiency, and relevancy); on 
the basis of the degree to which programs reflect community (employer needs),. state, and federal 
interests and concerns; relevancy to social and labor market conditions; and efficiency in operating 
the best program with the least ainount of resources. Current and projected economic conditions 
of rising inflation w^ith high unemployment point to the need for valid evaluative information to 
supjDort resource allocation decisions. 



Status of Employer ^tisfaction Assessment . 

In recent years, state and local agencies have conducted nunierous evaluations: . However, some 
of these efforts have'been piecemeal, and there has been minimal effort to develop and implement 
statei:and local evaluations which are comprehensive, continuous, and systematic. Nevertheless, a 
numter of states such as Alabama, California, Colorado, Illinois, Maine, New Hampshire, Oklahoma,, 
and Ohio have developed extensive plans for implementing evaluation efforts. Still, these agencies 
continue to face human and financial resource problems, in addition.to the complexity of evaluating 
misthodsand practices. There is^a continuing need for inquiry concerning the most appropriate evalu 
atjon strategies, procedures, and techniques to use at the state and local lej^^^ 

The problem is one of developing strategies, procedures, apd tecmiques that will offer account- 
ability and result in more meiapingful program improvement. David (raSQ), in a 'national study of 
vocational education, asserts that by the school year 1979-80, muclpctivity had taken place and 
strong commitment.to evaluation was evidenced ([d. v-I 6). However in a recent*case study of iFifteen 
state departments t)f vocational education, and in an analysis of a lithited set of documents from 
forty-one additional states and territories, Bueke et al. (1979) indicite that formal employer follow-up 



data are collected much less consistently than student follow-up data (p. 99). They add that approxi- 
mately one^tialf the case-study states do not have statewide employer follow-up procedures at both 
the secondary and postsecondary levels. This finding seems to show that vocational educators are 
less inclined to see employer follow-up as an important and necessary indicator of vocational program 
effectiveness (Bueke etal. 1980, p. 99).. 

Nolfi et al. (1978) offera detailed analysis of variables affecting work success in their examination 
of selected research findings. Citing the National Longitudinal Survey (NLS) of the High School Class 
of 1972, they indicate that much of the process of school choices and work success is currently 
unmeasurable; that luck, random influences, or factors that are beyond measurement with study 
variables have a very strong impact. They found that high school characteristics appear to have 
minimal impact on the success of graduates in the labor market. By comparison such characteristics 
as family income or socioeconomic status have a measurable ef|ect on labor market success, whereas 
ability (as measured by SAT scores or ETS tests) is jess important (Nolfi et al. 1978, p. 2). Additionally, 
they state that, personality and motivational factors appear to have some influence. Others have 
presented similar and opposing views regarding the measurability of factors affecting'school and work 
success (see Averch 1972; Darcy 1979; Grasso and Shea 1979; Jencks 1973, 1979; McKinney etal 
1978; Mertensetal. 1980; Pillimer and Light 1980). > 

There is some agreement among vocational educators that school systems should be held 
accountable for imparting certain knowledge and skills, and for making certain that there is a good 
fit between what is taught and the jobs available (David et al. 1980, p. v-5). Others find that many 
vocatio nal educ ators stated they believed their accountability should be for employability but not 
employment (Uavid etal. 1980, p. v-5). However, definitions of employability, like the word 
"outcomes,*^ havedifferent meanings among varied constituencies. 

It appears from a review of the literature (see McKinney et al. 1978; Stevenson .1978, 1979; 
Darcy et al.'1979) and discussions vyith certain SEAs and LEAs that each constitu6;ncy has its ovvn 
goal(s) for vocational education. Stevens (1979) ^dds that the interplay .of these and other group ^ 
interests determine how, when, in what form, and with what success particular goals surface in a 
recognizable way (p. 2). . : 

■ '. ' ' • ■ ■ ■ • ■ 

oln part, the many methodological problems associated with evaluation research highlight.the 
cQhtinuing search for effects of vocational education on employment. Copa and Forsberg (1980) 
state that: ' - " - ■ ^ ■ 

. ..a call is made forra Statement of specific prior expectia'tions for ' 

secondary vocational educatiba; with a'ccompanying rationale, whiclr 

could better serve as a "benchmark" ior judging the adequacy of ' . V 

those' effects which can be measured (p. v). . 

To answer, tfie;question, does it make ^a difference in accordance wjth canons of good research, 
is still a goal to be achieved. Moreover, in assessing the employer satisfaction with both the training 
and. job performance of former vocational students, the assessment requires adherence to valid 
procedures and methods. 

* Importance of Employer Satisfaction 

As mentioned earlier, one measure of the effectiveness of vocational education defined in the ^ 
1976 Amendments is student employment success. The regulations for the. 1976 Amendments 
suggest four factors of student employment success: (1 ) rates of employment and unemployment, • 
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(2) wage rates, (3) duration of employment, and (4) employer satisfaction with the per; manre of 
vocational education students, as compared with the performance of students who havener had 
vocational education [Regulation 104.402 (c)] . In addition to these federal requirement other 
reasons for assessing employer satisfaction have appeared in state and local evaluation studies, Asche 
and Vogler (1980) summarize reasons for the assessment of employer satisfaction: 

- • Recentemphasisonaccountability has created pressure to support the value of 
vocational education. 

• Employer input can be used to heighten ihtersst in vocational education and give 
students a better understanding of the programs. !, 

• Gathering employer opinion can be an effective means of setting priorities for 

• : program improvement. 

• Assessment of employer satisfaction can result in improved relations between 
vocational education and industry. 

• Information from employers on job content and requirements can.be very helpful 
for the guidance and counseling function, ' 



• Employer feedback can assist vocational educators in providing a labor force 

capable of adapting to new. technology and contributing to the development 
of this technology, " ■ ■ \ ■ . I 

' A list of key outcornes was idB3tifiedj±!rough""a small nonrandom sample of nineundmiduaJs 
knoyyledgeable in vocational education and experienced in program evaluatidn (Darcy^Hl..ig79 
p. 8). These individuals were asked to rate fifteen outcomes previously identified by prE^rstaff ' 
as being significant in thennselves and as'-having special importancis in evaluation. One ofihe fifteen 
key outcomes was ''satisfactorinessio employprs." the outcome statement was translalsd into an 
outcome question: how 'do employers rate former vocational students as compared With comparable 
nonvocationa! students in terms of attitudes, abiMes,'and performance on the job? The importance 
of employer satisfaction as an outcome was given additional support in other surveys (see Abramson 
Title, and Cohen 1979/p.,93-1 09). 

A major assumption undergirding this handbook is that any attempt at a valid measurement of 
employer satisfaction must include both the assessment of vocational training and job performance 
of the former vocstiorial students. Its value for program improvement is supported by many vocational 
educators; however, the most valid definition(s) and best methods of assessment still remain to be 
determined; ^ o ' ^ . 

- • : Summary - 

■ ■ w 

Tflis section highlighted the legislatgsffi.requirements and other reasons for state and local voca- ' 
tipnal education evaluation, including erapdoyer satisfaction as a vocational education outcome. 
Concerns about measuring vocational edrzcation effects or outcomes were presented along with a 
definition of external factors, such as pesonal motivation and economic conditions that create 
difficulties in obtaining inforrriation for^aluating vocational education onthe basis of outcomes. 
Jhe importance "of employer-^satisfactiominformation was summarized under'five reasons which 
support vocational education improvement and accountability. The potential benefits for assessing 
employer satisfaction^:are listed in the cbecklist that follows. ' . . 



CHECKLIST 2 
^ tent^al Benefits of Employer Satisfaction Feedback 



Purposes 

1. Provides information on employer satisfaction 
• for accountability purposes. 

2. Employer input can be used to heighten interest 
in vocational education and give students a 
better understanding of programs. 

3. Gathering employer opinions can be an effective 
means of setting priorities for program improvement. 

4.. Assessment of employer satisfaction can result in 
improved understanding between vocational 
education and industry. * 

5. Information from employers on job skills and 
attitudes expected of employees can be very 
helpfuiil for the guidance and counseling function. 

6. Employer feedback can assist vocational educators 
in providing a labor force capable of adapting to 
new technology and contributing to the .. 
development of this technology.. 

7. Employer satisfaction information has potential 
for aiding in addressing the issues and problems 
deating with productivity and reindustrialization! 



Rank Order of Importance 



CHAPTER III 

ISSUES AND PROBLEMS: MEASURING EMPLOYER 
SATISFACTION WITH THE TRAINING AND JOB PERFORMANCE 
OF FORMER VOCATIONAL EDUCATION STUDENTS 



Background 

This chapter discusses several issues and problems involved in measuring employer satisfaction 
with vocational education training. It is not an exhaustive. list; and many of these issuecand 
problems relate to activities in survey research and evaluation. Readers are encouraged to reflect on 
these concerns as they relate to their own experiences in doing employer foNow-up. 

The contents.of this chapter were derived from numerous sources, all based upon evaluation 
research, theory, and practice as well as personaj experiences of individuals responsible for the 
-design and conduct of employer follow-up studies. 



Source of Funding 

No extensive search is required to find a variety of follow-up studies of graduates; assessments 
of proigrams, employer studies, and the like. Many of these studies have been conducted by outside 
agencies under contract for a specific purpose. In many cases, the source of funding for the study 
(whatever it may be) is fr'om yet another agency-a parent agency, a state department, a federal 
office, or a consortium. . ~ 

•^ss often'been said that educators live-by the golden rule-they who have the gold rule! If 
this glib point of view carries any message atall, it is that as projects and stiijrties are funded, they are 
accordingly subject to the dictate's of their financial supporters. . ^ . , 

i '. . As fjjnding comes more and more frohi a centralized source, it follows that it will be accompanied 
by a mandate for particular emphases. The satisfaction of employers of vocational education graduates 
-essentially a grassroots mattar-becomes increasingly subordinate to the whims and fancy of the-' 
funding source. When fundrng:so?jrces become more and more, centralized, a question generally arises 
- * as to whether the needs of local employers, and.concomitantly, ,the ioca! vocational education pVogram 
are being met in the most expeditious way. 

^ If the satisfaction of employers of vocational education programs and former students, therefore, 
is to be assessed from time to:time, provision must be made for such appraisals to occur rggularly. 
SbmiB priority must exist to:assure measurement pf the opinions of the smallest of employers as well ^ 
,as of the giants; provision must also be made for freedom ih designing the most appropriate method 
of sepuring the best possible data at all levels of. the vocational education spectrum. 
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Prbgrams to Survey 



Researchers and practitioners alike are mixed in their attitudes and opinions regarding data 
collection. Some view a mixed approach as appropriate, so that all constituencies are involved 
whereas others strongly prefer selecting segments of the total population and dealing in greater- 
depth within each of those segments. 

Deciding which approach to use conducting employer assessments differs little from that of 
other evaluation research projects. A variety of vocational programs exist in any given site, each with 
a number of students whose employers are sources of information to the study^. 

If a particular program is in need of information on satisfaction measurement, then the question 
of which to include is easily answered by selecting employers who are known to have hired graduates 
. of those programs. If such is not the case, the question can be answered by how extensive the study . 
IS to be and which programs, if any, can be eliminated from the assessment. 

^ .. • • . ^0 ' 

One way o guiding the decision about which programs to include or to eliminate is illustrated 
in a study (iSmiley 1976) in which an initial data base was derived from all seconder/ and postsecondary 
sites in North Dakota with five or more vocational programs. The number five was chosen arbitrarily 
and could easily have been changed upward or downward as deemed most appropriate. By using a ' 
similar technique, specific programs could either be eliminated from data gathering if the number of 
student participants or graduates fell below soine predetermined quantity, or added if the number 
.exceeded some figure. 

As with all effective research, the sample selected for study is important, if not crucial, f6r the 
purposes of generalizability. This and sound early planning are among the most important ingredients 
for satisfactory culmination of any study. 



Variables introduced by Selection of Subjects 

Vocational education programs exist in both^secondary and postseconda'ry set^ ■ 
enrolled in and graduatinig from these two sett>fgs enter the job market armed with somewhat 
^ different^skills, quite different levels.of maturity, a factor certainly not to be discounted. 

Cell and Jones (1975) found that employers were more inclined to Hire graduates with associate 
of arts degrees;ffor instance, than those wrthlesser accomplishmeints. Srhiley.found (1976) that 
employers view older/more mature^students.as more satisfactory employees than younger, less 
mature ones. Yet in a survey of three large private companies (aggregate employment of approximatley - 
one mil!ion)-one in manufacturing, anb'ther'in utilities, and a third in retail tra..de-Ginzberg (p. 40) 
fouhd that, "generally speaking, there were no major differences between young workers and adult 
workers in the objective data provided regarding on4he-job performance' (1976). ' 

A question arises, therefore, about the type of student to be used to ascertain employer satis- 
faction. If employers of students^who are more mature and better trained are used, a somewhat 
Nuritrue picture of their former program may result. Purposes of the assessment should determine what 
sbWects will be selected and what variables will be included. • 



The Self-employed Graduate 



..Although it is a rather easy matter to identify employers of vocational education students and 
graduates, it is somewhat less easy to obtain a clear understanding of their satisfaction with these 
employees. A predicament yet more troublesome exists when information is sought from self- 
employed persons who are graduates of vocational programs. Several conditions cf self-employment 
illL/strate this problem. 

First, graduates who are self-employed (especially within a few, years of their exit from the 
program) have typically not had reason to maintain a close working relationship with those involved 
in their former program. The absence of such contact, therefore, creates a void which tends to work 
against easy identification of that particular segment of graduates or former students. 

Second, when self-employed persons are identified, they form a constituency that carries a dual 
role-both student/graduates and employers. As both employers and employees, they cannot be 
expected to respond to many of the questions relating to hiring graduates and the adequacy of 
relationships between employer and employee. It may be necessary, then, to produce a separate set 
" of questions in order to elicit the.same relative infomi 

Third, self-employed graduates of a vocational education program do not have the benefit of 
. being able to make comparisons with other graduates or other employers. Their position, unique and 
incapable of being duplicated, does not admit itself to generalizations, since all responses regarding . 
satisfaction are specific and totally subjective. ' - 

Fourth, self-employed graduates may be more lil^ely to have responsibilities riot directly related 
to their vocational education programs. ^ - : . 



Time Lapse Between Training Survey and Employer . 

Vocational education programs undergo changes-both Bvpju|^ and revolutionary; As these 
programs fluctuate, and as personnel changes occur, a reasonable fexpectbtion Is that responses 
regarding satisfaction may likewise difW. • / \ 

Longitudinaf studies often indicate an ebb and flow in many aspects, w[hich rriay be desirable 
from time to time. Because of this apparent reality, a potential danger is present if too long a time 
sp5n elapses without identification of the time categories. The reader is encouraged to review the 
following sources for a more detailed discussion on measurement problems using a longitudinal design 

^ S: J. Franchak et al.. Specifications for Longitudinal Studies (Columbus, Ohio:. : 

National Center for ReseartiH in Vocational Educatiohf the Ohio Stat^. University . 

1980), pp. 25-^19. . : " • 

' i^.Go\dste\n, Tfie Design and Anafysis of Loh^ , 
, Academic . Press; 1979),* pp. 4^5^ 50-69. > \. 

. ^ • • ■ ' • ■■ '-^^ : • >■ 

J. R. Nesselroade and P. B^. Baltes, editors. Longitudinal Researcfi in tfie 
Study of Befia\/ior and De\/efopment (New York:, Ac^dervi]^ - 
pp. 29-39. ^ ' \ 

■ i. . * . ■ ■ 

One of^the complexities of fusing academic pursuits of ddta collectidn, Vesearch, and program 
planning and delivery with the more practical aiDplications in business and industry is the different 



perspectives of the two camps. Business and industrial employers are concerned with productivity 
^1^'?'®"^^' P''°^'^ ^^^'^^ enterprises. This concern emphasizes immediacy and generally 
dictates that current conditions are moreJmportant than what was occurring four or five years ' 
earlier- or what may be occurring several years in the futufeT — . ■ 

Educators, in their zeal for appropriate information about program evaluation and planning 
must be aware of the more immediate focus of employers regarding productivity and profit, if 
cooperation is to take place. Data from employers who are aware that their firms will ultimately 
benefit from evaluation research effort will more likely provide the evaluation agency with accurate 
employer -satisfaction information. 

Limitations of Training Programs ' 

The content of the curriculum, th'e types of experiences students encounter^nd the overall 
philosophic stance of personnel invoiyed in a program are all determining factors of whether vocational 
students will be job ready when the^ complete a program of study and move into the job market: 

. Employers often react favorably regarding the extent of skill development that vocational 
- education gra^^ wherl coming into the workforce, but are frequently less pleased with 

the lack of more general education skills of ^many 8mployees,''Who hope to be promoted to positions 
with greater responsibility, prestige, and income. ^ . 

An example of this point was reported by AnniS and Perrigo (1968). They found that employers 
constantly faqe a problem in securing supervisory personnel from within the ranks of employees. 
Althpugh many of these employees possess the required specific vocationarskills, they are unfortun- 
ately lacking in the so-called leadership skills and human relations skills necessary for these supervisory 
positions.-Consequen^ly, the question that must be considered^is whether Employers can, in fact, he " 
totally satisfied with employees who do not bave the wide variety of skills accessary to advance in- 
more complex jobs. 

. • ■ . ^ * ■ : ' — \ . ■ 

. * ' Cost Factors of Survey Methods ^ ^ ' 

It is important to analyze the cost of various survey methods before embarking ^on a study, a 
choose the methods whose cost fits your budget. It is an unfortunate fact that survey and study 
priorities cannot always be carried.out within the constraints of limited financial resources. . ' * 

Educational institutions are constantly involved in the collection of data of one kind or another, 
and methods of securing data vary considerably 5mong;differentevaluato^^^ ' . 

it seems, is the majled questionnaire; but seldom do two people agree on an expected rate of return/ 
, - Follow-up postcards as reminders.usually result in greater retiirns; telephone remi^ likewise. 
Still others wish to engage in tremendously sophistioated coding procedures in order to determine . 
,who has responded and who has not, at the same time assuring the respondent total anonymity. 

' • ■■' . ' ■ " • I ■ ' ■ -■■ ; ; ■ - ' 

, Educators, asa group, can typically be«expe'cted to respond to mailed^ questionnaires in excess 
of 60 percent. Business people, on the other hand, fall far below that figure. Peters (1977) was able, 
to secure responses from only 32 percent of 480 organizations in a mailed questionnaire survey, 
■whereas 191 of 226 from that same population agrepd to and held personal interviews in an feariier 



study (Smiley 1976). 

"■ ■ ■ ' ■ V 1 

■ ..V .... - - \ . 
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Efforts toward public relations can certainly be enhanced when personal contact is conducted 
unobtrusively and sincerely. Mailed questionnaires, because of their impersonal nature, often produce 
negative as well as positive results. The expense of traveling great distances to hold personal interviews, 
however, works against conducting such studies, especially in those geographic areas where graduates 
are widely scattered and not readily accessible within close proximity. In this case, alternative means 
of data gathering must be considered. 

The use of the telephone interview is certainly a viable compromise between the personal, 
face-to-face contact and the more impersonal mailed questionnaire. Costs are decreased considerably 
through elimination of traveling, lodging, and transpprtaiion^costs. For further discussion on follow-up 
costs the reader is encouraged to see: Guidelines and Practices for Follow-up Studios, by S. J. Franchak 
and J. E. Spirer 1978, pp. 101-104; and Mail and Telephone Surveys by D. A. Dillman'(New York: 
John Wiley, 1978), pp. 68-72. . 



Role of Advisory Committees 
in Measuring Employer Satisfaction 

Vocational education programs utilize advisory committees in a variety of ways, involving 
members in the assessment of employer satisfaction is certainly an appropriate use of their individual 
and poMejctive expertise. To overlook this significaht and important resource is to weaken the impact 
that advisory committees may have on the effective. functioning of programs in vocational education. 

The selection of members to serve on these committees is crucial> since what each individual ^ 
brings to if either enhances program growth or contributes to its deterioration. Having advisory .. 
committees comprised 6f intelligent, articulate, and forward-looking people offers virtually unlimited 
potentiaTfor the attainment of quality programs that result in highly satisfied employers. Moreover, ' , 
their help with evaluation activities can be invaluable. - ; ' . ' 

' . . ■ ^ • ' ■ ■ ; ' : " ''. : • 

Differences in Emphasis within Training Institutions ' 

nrhe employee who has completed a program at the employer's training school may develop ^\ 
a camaraderie with the employer that colors the-nemployer's^satisfaction. Conversely, an opposing 
-point of view or a less satisfactory relationship may develop between the empfoyer and an employee 
who has completed a program at some other, or rival/Institution. 

By the same token, employers who have themselves com^pletisd a particular prograrh are often 
unconsciously more favorably disposed toward, eniployees vyho have gone through' the same program, 
than toward those who^come from some otfier^area of training ' 

Face-.to-face encounters contribute considerably in providing opportunities to observe this 
phenomenon and to\deter mine more accurately how the, satisfaction with the employee is associated 
with the place and area of training. . 

Evaluators who have the. opportunity to observe face-to-face encounters with employees and 
erhplqyers are obviously in a better position to judge this phenomenon and to gauge more accurately 
how satisfaction with the employee is associated with the place and area of training. \ ^ 



Researchers and data collectors are intimately familiar with sampling techniques and the 
potential problems created by not following random selection procedures. Data are necessarily 
suspect when samples of employers are "selected." ratfier than "randomly selected." A thorough 
reading of many surveys will raise the question as to whether data were collected from employers ^ 
previously identified as favorable toward a given program or institution. 

When instructors and administrators are asl<ed to suggest employers that have hired graduates 
there is a temptation-perhaps a tei^dency-to overlook the known skeptics. If such an incidence 
takes place, a potentially valuable source of information is thereby eliminated. 

. ' A later stage in the collecting of data finds the nonrespondents creating a void potentially 
damaging to data validity. Samples of employers need to be drayyn that will eliminate such voids, 
^ and methodologies need to be used that can.turn nonrespondents into respondents. 

\ . j Employer Contact 

It is obvious that the satisfaction of employers with graduates of programs is important to the 
* continued success of any vocational education endeavor. If employers gre satisfied, graduates may 
easily secure employment; if unsatisfied, employment may be difficult. If some degree of dissatis- 
faction exists, changes in curriculum, personnel, materials, and so forth .may need to be considered. 
Contact with employers cannot t>e overlooked, if students are to be well se'r^^^ V 

^ Several suggestions have already been put forth regarding how employers may be reached. An 
important element in such contact is to make employers aware that they are indispensable to the 
success of vocational programs, and that such success will ultimately pay rewardsjo^em. 

The significance of putting the best foot forward t6.employers,'a goad p/ublic relations technique', 
cannot be overestimated. If employers are made to feel irhportant, asked for advice and counsel, and \' 
-looked upon as a valuable resource for program development, the vocational education program' will 1 
more than likety benefit in return. \ 

Employers are busy people who have businesses to manage, factor^ to operate, and a myriad ^ 
of activities to oversee. If that work is not: done, productivity will decide and profits diminish; All ' 
of these are mentioned to underscore the importance of maintainingi- clear direction, keeping appoint- 
ments p/omptly, avoiding idle talk, anX meaningless questions. Conducting the business of vocational ' 
education judiciously and efficiently carUjave^positive payoffs not always overtly identified. 

* . ■ ' ■ '■ ^ ■ ■ ' . ■■■■ 

In instapces where s^everal employers are being surveyed, it becomes readily apparent that they 
are not all alike in their expectations. Getting to the point of knowing what employers' expectations':, 
are is important in understanding their level of satisfaction. ^ 

* " ■ ' • ' ' ' " . ' 

Questions should be designed to elicit the employer's background 'of training and experience, 
type of organization represented, and expectations for employees. For example, it is important, . ' , 
perhaps, to know that a particular employer is responding from the perspective of believing that 
every high school student shbu.ld be required to study. Latin. Naturally, ithat employer's response 
may be quite divergent frofffthe response of one who finds the entire area of language development 
useless.' 



How to pose and phrase questions to reach this level o'f sophistication is demanding, timi 
consuming, and otherwise disagreeable. It is, however, potentially crucial for gaining a precise 
understanding of what employers of graduates are saying. , 



Analyzing and Reporting Data ' >• 

Interviews frequently elicit much off-the-record information and information not requested, 
m addition to the considerable data.that are sought; whereas questionnaires that are mailjsd and 
returned sometim^'es omit valuable messages. Regardless of the method lised to Secure information 
orthetypeof information re.ceived, the reporting of it is enormously important. « . 

Numbers do not lie. What those numbers say and how they are organized may. lead to significant 
conclusions that ultimately bring about recommendations of great importance*; 

Employers' negative attitudes and opinioQS expressed in surveys haVe a tendency to surface 
before they can be properly analyzed and reported in context of the whole study-sometimes with 
damaging results such as hurt feelings, defensiveness, and even retaliatory action. Therefore great )^ 
caution should be exercised during this stage of the information processing to maintain confidentiality 
, of responses. . «a ' . ^ 

Formal reporting of negative survey findings thatibecome quite specific in'their analyses may 
' be quite beneficial to the total program and organization,, even though they may. bring about personne 
and program disapppintmehts. In the long run, knowledge of this ostensibly negative outcome may be 
necessary to generate needed improvement. ) 

A ' ■ ' ■ .... > ■ - ■ ' • ■ ■ - <^ . ■ "i' ' ■ ' ■■■ ' . " " ' , 

•"J . ■" , ■ . 

On the other hand, when all results are positive and personnel and. programs are reinforced, the^ 
end result may be an increased sense of worth and improved morale. ' 

, Having outside agencies or persons conduct infQrmat^^n gathering may serve the org^nization 
well. The outside agency has no vested interest and. can perforrn m ah.unblased maniier. Reporting'pf 
Results follows the same pattern, and since the outsiders do hot have to 'live with" mernb^e 
organization, greater candor can be expressed. Animosities thgt may develop are directedtto outsiders) 
ahd relationships within'the organization may be. better preserved. , ' 

Contracting outsiders to engage in data collection and reporting may detract somewhat from ^ ■ ' 
desirable public realtipns aspects but, at thfe same time, it offers potentially better opportunities for 
higher quality data analyses and reports: Typically, the expense of contracting with outside agencies ; . 
will .be greater; but such an approach has the advantage of not adding to.the normal work loads of 
those who would otherwise b^l.occupied by cpnducting'the s^^^ . ■ 

^. Althdugh.the various issues and problems surrounding the matter of soliciting employer satis- 
faction with the^graduates of vocational education are numerous, they are not insurmountable. Most^ . 
can be dealt vyith when foreseen in advance and considered in the planning that takes place prior to 
information gathering. Those \A/ho work with this type of project regularly would be likely to say 
that "common sense" dictates what should be dpne. Unfortunately, /'common sense" is Useful only 
when expertise and basic knowledge or understan..^^ng are applied in appropriate'dosages. 



Summary 

The importance of the employers of vocational education graduates in evaluation cannot be" 
overemphasized. They are Immensely valuable to any program; periodically'^nd frequently'they 
must be contacted in order to gauge their satisfaction with the products of vocational education 
programs. The products of vocational programs are an integral part of the resources of the enterprises 
which employ them. Satisfaction with those products can be compared to satisfaction of a consumer 
with any product placed on the market, - 

In order for vocational education programs to market their products adequately, surveys of the 
consumers are a necessary part of the total effort. Well-considered and methodically prepared efforts- 
should result in meaningful assessments, if attention is given to those issues and problems outlined 
in this chapter. 
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CHAPTER IV 

EMPLOYER SATISFACTION WITH ^ 
^ TRAINING AISID JOB PERFORMANCE 

' i ' , • ' . 

This chapter identifies and defines differences and commonalities between employer satisfaction 
with training and satisfaction with job performance of former vocational education students: No 
attempt has been made to address all aspects of those differences and commonalities. Rather the 
intent is to highlight only those differences and commonalities that vocatibnalj evaluators should 
address when doing employer follow-up. Specifically, this involves the need to accurately define wnat 
data and information reflect the evaluators' follow-v»p study objectives. In exajnining some employer 
follow-up efforts it was found that often the distinction between employer satisfaction with training 
and with job performance was not defined. This chapter offers information that should help the 
vocational evaluator make that clarification. 



/ Training Satisfaction and Job Performance 

^ ■ 

Training Satisfaction 

Finding a consistent and valid "working" definition of employer satisfaction with training has 
been at best a varied and difficult problem. Moreover, there appears to be an emphasis on including 
satisfaction vyith job performance, and satisfaction with training as„shown in various employer 
follow-up efforts. For example, the National Center for Educational Statistics' Vocational Education 
Data System ( VEDS) i-equires that state and local education agencies collect data on vocational 
program completers and leavers to determine if their ernployers considered them to be well-trained 
and prepared for employment. This is defined as an assessment of the performance of vocational 
education sturients as compared with performance of persons who, have not had vocational education. 

Figure 1 lists questions required by VEDS in this employer assessment. 

J. ■ ■ ' . ' ■ - ' * 

' In a national review (period of 1970 to 1977) of follow-up instruments, O'Reilly and Asche 
(1979) found that almost all assessments asked employers to rate each former vocational education 
student on selected work' skills, habits, and attitudinal constructs. Table 1 shojys the number and 
percent of the employer follow-up instruments reviewed by data elements found in selected questions. 
These findings corroborate. those of Gray et al. (1978) who.also conducted a national review of 
follow-up studies focusing on data elements associated with job satisfaction, student satisfaction 
vyith training, and. employer satisfaction. 

Other references to employer satisfaction focus specifically on the vocational program or 
training. Taken literally, this would imply that the employer will evaluate the vocational curriculum 
rather than the on-the-job performance of students. The use.of this definition presupposes .that.tlie. ._ 
employer doing the evaluation has detailed inforrnation oh the curriculum and will make an on-site 
review of the vocational program. Very seldom does this occur in a forrrial sense. 



FIGURE 1 
VEDS Employer Satisfaction Form 
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• - EMPLOYER FOLLOW-UP.FORM - ' 

Note: This report is authorized by law (20 USC 2312 and 20 USC 2391). While you are not 
required to respond to this survey, your cooperation is needed to insure that the results 
of this effort are comprehensive; reliable, and timely. No student fdentifiers will b:^ 
forwarded to the Federal government. 

1. Vocational Training Evaluation ™ 

RIease rate the vocational training received by the individual in the following areas: . 



a. Technical knowledge 

b. Work attitude 

c. Work quality 

dverall Rating 

What is your overall rating of the vocational training received by this individual as it relates to 
the.requirements of his or her job? 

Very V^yy 
Good Good Average Poor \ Poor 

• I ' ^ . 3 ,, 2 1 

Relative Preparation , ' 

Asa result of tl^is person's vocational training, how would yourate his or her preparation in. 

relation to other employees in his or her work group who did not receive such traininq? . 

\ • 

i . No basis for comparison 

(5) Individual is better prepared 

(3) Both are about the sanie: 

n) Individual is less prepared ' 



NCES Form 2404, Page 1 of 1. This report is authorized by PL 94-482 (20 U^C 2391) 
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TABLE 1 

Employer Fo|lpw-Up Instruments by Data Element 



.. ■ ■ • — 
Data Elements* 


' ' No. 


Percent 


Percent Quality of Work 


16 


'84 


Quantity of :Worl< 


14 • 


% ■ 


Job-Related Technical Knowledge 


13 . 


68 


Job Skills 


10 


' 53 


Operatiorr of Tools and Equipment 


11 


59 


Work Habits/Attitudes . 


8 - 


42 


Accept Responsibility 


10 


53 


.. Punctuality 


~ 8 - . 


42 


Supervision Required , 


10 


53 


Cooperation with Coworkers 


15 


,79 


Willingness to Learn and Improve 


/. 14 


74 


Cooperate with Management , 


1 1 


59 


Compliance with Roles/Policies 


8 


42 


Attendance , v 


12 


' 63 


Overall Satisfactoriness 


11 . 


59 


Promoitability 


. 4 


21 ' 



Note: Nineteen instruments comprised the sample. 

*Data elements refer to the employer's rating of the employee on the characteristic. 
Source: O'Reilly, P., and Asc^e, F. Follow-up Procedures: A National Review. Blacksburg, VA: 
Virginia Polytechnic Institute and State University, 1979 



Job Performance Satisfaction* 

^ Job performance is a very complex construct and is not considered to be one concept or trait 
but rather a cluster of dimensions. These dimensions have been labeled in various ways, such as level 
of skills, quality of work, quantity of work, technical kno.wledge, attendance, punctuality ability 
to work independently, cooperation with coworkers and superiors', communication skilis, problem- 
solving skills, interpersonal skills, safety, initiati\?e, attitudes, and so forth. Many would agree that 
there is no one correct list of job performance skills (McKinney, Gray, and Abram 1978; Smith and 
Brouwer 1977). The ideal approach is to do a complete job analysis for a specif ic job and design an 

: instrument for that job. i 

. Jn relationship to training satisfaction a valuable activity would be to analyze the competencies 
taught and develop an instrument for that training program. Ideally, there should be a'high correlation 
between the actual tasks on the job and tasks defined in the job analysis. Most often, a small set of 
general dimensions are identified and applied to all jobs. A compromise is to select an instrument 
specif icalSy developed for the job being assessed or a similar job. Another alternative is to select an 
instrument that^contains dimensions considered to be meaningful, as judged by someone familiar 
with the job. ' 

A model of job performance, ^ob performance is generally thought to be determined by three 
basic variables: (1) motivation or effort, (2) skill level, and (3) role conception. Effort (how hard the 
employee works) and skill level (whether or not the employee has the skills to do the job) are self- 
explanatory. Role conception refers to the employee's idea about what should be done on the job 
and how the role should be played. If an employee has an incorrect notion of how the job should be 
done or what activities constitute high performance, then actual performance will be low. Even though 
the employee may have the abilities and be working hard, performance suffers if effort is put into the 
wrong activities. " 

Each of these three determinants 'has a unique and crucial effect on job performance. That is, if 
one variable is very low, then performance will be jow. Total lack of job skills means low performance, 
no matter what the motivation level or role concept. In order to analyze the possible effects of voca- ' 
tidnal education on job performance, we must determine which of these three factors might be altered 
by vocational training. Figure 2 depicts the factorial relationships of job performance. 

Motivation and vocational education. There are too many theories of work motivation to 
summarize all of them here. However, expectancy theory and goal setting are two major approaches, 
to motivation that dominate current thinking and provide a basic framework for analysis suitable for 
measuring job performance. Each is brief ly^discussed. ^ 

At present a dominant theory of work motivation currently is expectancy theory (see Campbell 
and Pritchard 1976, for a comprehensive review). According to this thepry, employees are motivated 
to perform if effort is seen as leading to work outcomes, and if ^hey consider those outcomes desirable. 
Work outcomes may include extrinsicoutcomes (e.g.,. pay, promotion, relations with coworkers, 
working conditions) or intrinsic outcomes (e.g., autonomy, feeling of accomplishment, pride in work, 
enjoyment of the work itself ). 



Adapted from R. S. Billings, "Job Satlsfactron and Performance Measures: The State-of-the-Art", In Interpreting 
Outcome Measures in Vocational Education: A Final Report, by F. L. McKinney, Kr E. Gray, and M. Abram . 
(Columbus, Ohio: National Center for Research in Vocational Education, 1978), pp. 155-161. 



FIGURE 2 



A Model of Job Performance 



MOTIVATION 

(Perception of Effort) 
(Value Placed on Intrinsic butcomes) 
(Setting Performance Goals) 



JOB 

PERFORMANCE 



(Aptitude) 
(Traimng) 
(Job Experience) 



ROLE CONCEPTION 

(Employee's Ideas 
About What the 
Job Involves) 



Gray, K.E. et al. Vocational Education Measures: Instruments to Survey Former 
Students and Their Employers., (Columbus, Ohio: The National Center for Research 
in Vocational Education, August 1978) p. 266. i 



Thus, vocational education could increase motivation by affecting any or all of these expectancy 
: theory componetns. First, the perception that effort leads to performance might be affected. As 
students learn how to do the work and become more confident, then they will begin to see that.high 
performance Can be attained if 'the effort is made. The perceived relationship between performance 
and work outcomes seems less amenable to shaping by vocational education. This perception is affected 
primarily by the reward system in the organization itself; some organizations reward performance and 
others do not. The final variable, the value placed on work outcomes, might be affected by vocational 
c education for some outcomes. The value, of intrinsic outcomes (e.g., autonomy, pride in work, feeling 
of accomplishment) seems mofe likely to_be affected^han the value of extrinsic outcomes (e.g., pay, 
praise), in summary; expectancy theory,.might predict a slight effect of vocational education on intrinsic 
motivation to perform, if it increased the perception that effort leads to performance and the v^lue of 
intrinsic work outcomes. - - ' — 

- . , ' ■ ' ./ . • V""" ^^-^ • 

A slightly different approach to motivation has been advocated^t)y Locke (1968). He suggests 
that performance is determined by the goals a person sets for oneself. Further, the more specific and^ 
the more challenging the goal, the higher the level of performance. Finally," the individual must fully - 
accept the goal before it will affect performance. 

Using this approach, vocational education may increase motivation and performance by altering 
the performance goals of the individual. More specific goals may be acquired during training as the 
student is taught exactly what doing a good job entails. More challenging goals may emerge as job 
skills and aspiration levels increase. Goals will be accepted more^completely as the student understands, 
the reasons behind performance goals and identifies with the occupation. ' ■ 

In summary, vocational education might increase motivation and consequently job performance 
by (1). increasing the perceived probability that effort leads to performance, (2) increasing the value 
placed on intrinsic rewards, and (3) heightening the performance goals set by the individual. Unfortun- 
ately, many other, variables also affect motivation, such as job characteristics, supervisory style, organi- 
zational reward systems, value placed on extrinsic rewards, and vyork group performance norms. Since 
these are largely unaffected by vocational education, the overall effect of high-quality vocational 
education on work motivation may be moderate to nonexistent. 

Skills and vocational education. The second variable affecting job performance is the skill level 
of the individual. Vocational education is generally thought to have an impact on job performance 
primarily by affecting this variable. Although this may be true, the situation becomes more complex 
when analyzed closgly. * , 

Lawler (1973) has suggested that job skills result from underlying aptitudes, training, and direct 
job experience. Because aptitudes are generally considered to be essentially stable characteristics, 
vocational education would not be expected to affect job skills through aptitudes. Likewise, experience 
on the job has an effect on skills apart from the quality of vocational education. Even within the realm 
of training, vocational education is only part of the story. Important skills are certainly affected by 
the quality of nonvocatidnal education, hobbies, extracurricular activities, ^d other nonschool 
experience. All of this is to say that even though vocational education may affect job skills, it is'only 
one of many factors that affects such skills. Therefore, the effects of vocational education on job, 
performance through job skills may be modest and difficult to identify. 



Role conception and vocational education 



^ This is the least studied and least discussed of the variables affecting job performance. Yet it is 
quite different from the other factors and probably has a significant impact on job performance 
in rnany situations. It is not a matter of being motivated to perform well or ofhaving the necessary 
skills; rather, the idea about what the job entails may differ from the correct model or the one 
advocated by the organization. For example, the job performance of insurance agents may be rated 
low if they feel that their job is to serve existing clients and not to develop new. customers. 

Vocational education may have an impact on students' or graduates' roje. conception; part of 
good training may be.learning what the occupation ehtails-which facets are most important and 

-which are of lesser priority. If this. elFfecrof-vbcatianaLedu cation occurs, it is likely_:to_be^iTi.ost 

noticeable. in new employees. Assuming that most untrained employees only learn the proper role 
concept over time as they gain direct job experience, the graduate of vocational education who has 
acquired.a. proper role concept should be more able to step into the job and know with minimal 
direction what tasks should be done. . ' 

Conclusions. There are many determinants of job performance. This seemingly obvious point 
niust not be forgotten, for it implies that the effects of even vory good vocation'cil education will be 
difficult" to discern because of "error variance." If all the extraneous factors could be controlled (e.g., 
individual motives, job characteristics, organizational reward systems, underlying aptitudes, job 
experience, other training), then the effects of vocational education could be easily and unambiguous 
determined; In any reaMife study, these uncontrolled factors have such a large Influence that effects 
from the factors being investigated cannot be accurately measured. 

Several suggestions can be offered. The use of large numbers of subjects can help to tease out 
subtle effects. Whenever possible, the factor suggested by the model presented here should be 
measured and controlled. For example, comparing vocational education graduates with nonvocational 
education graduates within the same job classification, organization, and length of service would 
control many factors. ' 

On a brighter note, this analysis has suggested several possible mechanisms through which 
vocational education could affect job performance: increased miotivation, better job skills, and 
knowing what the role entails. Whether or hot these things do occur is, of course, an empirical 
question. However^ recognizing the possible causal mechanisms has two important applications. 

The program itself should be analyzed for existing effects, and possible improvements in the 
motivational, ability, and role concept domains. The first step is to examine the goals and conduct 
of the program. Specific job skills are certainly a central focus, but is it also the intent to increase- 
knowledge of the role and motivation? If so, the discussion above provides some specific guidelines 
for analysis (e.g.. Is self-confidence or perceived effort to performance relations enhanced? Are high 
performance standards encouraged? Is the importance of the occupation emphasized? Is autonomy., 
encouraged?). . 

A comprehensive^ useful evaluation of job performance of vocational education graduates will 
not be limited to measuring overall performance; items should deal with contributing elements of 
motivation, ability, and role concept. Whatever the findings for overall job performance, the adminis- 
trator will need Jo know why there was. or was not an effect on performance by these oth§r elements. 
Further, the implications for the content and conduct of the program can be known only if more ' 
specific items are included. Imagine that a program is evaluated by measuring the job performance of 
recent graduates and that they are found to' be performing no better than similar employees without 



vocational training. What should be done? A likely response woujd be to try to add to or update the 
content of the progrcim-that is, try to increase job skills. However, the problem could be low 
motivation of graduates (e^., the teacher lowered the students' self-esteem and their perception 

. that effort will lead to improved performance by constantly critici?ing) or role concept (e.g., the^ 
teacher has an idea of how the job should be done that differs completefy from that of the eventual 

, employer). In other words, evaluation of only-the overall job performance of graduates rnay actually 
be worse than no evaluation at all; without more specific information on motivation, skills, and role 
concept, the wrong problem could be addressed. 



Sumrnary , ; • 

- --.YQcati_gnal,^e^ employer follow-up information 

needs. Specifically, the determination must be made as.to their need for obtaMng Wt^^ 
satisfaction with training program(s) or satisfaction with the job performance of the former vocational 
education students, or both. Once this decision is made the evaluator must operationally define that 
variable or variables to be evaluated. .. . n . 

As discussed in this chapter, finding a consistent and valid "working" definition of employer 
satisfaction with training and job satisfaction has been at best a varied and difficult problem. The 
cluster of concepts or factors associated with "training" are varied. However, by examining the total . 
school curriculum and the school environment those concepts or' factors can be identified. Moreover, 
the measure of awareness or knowledge of those concepts or factors. by the employer must be assessed 
by the evaluator for reliability and validity. The cluster of concepts known as job performance are 
complex. However, by (1 ) measuring dimensions of performance which are important for a given job 
and (2) measuring the causes of performance-motivation, skills, and role concept, a useful and under- 
standable evaluation can be performed. . 
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Essential Elehrients of Information, for an Employer Follow-Up Study 



Checklist of Elements 



Information for this Outcome* 



1 . Outcome question to be answered: 



2. Outcome hypothesized as the ' 
dependent variable: 

3. Affected entity (student population 
or other entity in which the 
outcome is observed) : 



4. Program identified as the strategte 
independent variable: , 



5. Rationale for hypothesized outcome 
as a basis for evaluating the program: 



6. Empirical indicators of the outcome: 



7. Methodological considerations 
(evaluation design, sampling 
procedures, statistical analysis) : 

8. . Evaluation standard (basis for 

interpreting outcome data to ] 
. judge program effectiveness): 



9. Data requirements (instruments, 
procedures, data base): 



To what extent do employers consider former 
vocational students to be well trained and ^ 
prepared for employment? 

Satisfactoriness of vocational programs as 
perceived by employers. 

Employers of former vocational students (as 
influenced by performance of employees who 
~took-vocational-educatibJi)_,_ 



Secondary and postsecondary vocational 
programs that purport to impart entry-level 
job skills. 

To the extent that vocational programs 'pur- 
porting to teach entry-level job skills are 
successful, they will turn out program com- 
pleters and leavers who are likely to be 
considered by their employers to be well 
trained and prepared fonemployment. 

Employer ratings of former vocational 
students. (Legislatpfi specifies the use of 
employer perceptions as the required 
empirical Indicator.) ^ 

Sampling procedure as used in the state or 
local evaluation systfem. 



Higher ratings for, (a) former vocational 
students than for comparable workers doing 
the same jobs who did not^taj<e vocational 
education, or (b) the presenf cohort of 
vocatiohaUtudentsas compared with earlier 
cohorts,* ' . " I 

Employer responses to a mail sugvey at the 
time of student follpw-up. Questionnaire 
designed by SEA or LEA .^ased on national . 
VEDS, adapted to providejfor comparison of 
vocational students with nonvocatiohal, or 
supplemented with a second questionnaire to 
collect data on employees|without a voca- 
tional background. 



Table 2 (continued) 

' / Checklist of Elements 

"P^ — — 



10. Feasibility of investigating this I ^" 
outcome (conceptual, administra- 
tive,' cost, time, other considerations) : 



1 1 . Findingft (results of data analysis, 
warrangod inferences, 
generalizability): 



12. Potential impact of findings 

(implications for policy, program 
design and management, image 
of vocational educathDn): 



1 3. Dissemination of findings and 
suggested applications: 



14. Implications of findings of .further 
- RD&E activities and relationship 
of this evaluation to previous 
studies: 



Informatidn for This Outcome 



"^hewip loyerratingsTjff ormer^ I 

students will be easUv^and inexpensively 
obtained under established VEDS require- 
ments. Validity of responses may be 
questioned, including employer comparisons 
of vocational with nonvocational students. A 
separate survey of employer perceptions of 
nonvocational student's would be difficult 
and costly. 

High ratings given by employers to the training 
and job preparation of the employees'can be 
interpreted as a positive indication of voca- 
tional program success. If responses vary by 
item, school,. employer, industry, or occupa- ^ 
tional program, this can identify potential 
strengths and . weaknesses. 

Findings might demonstrate the need for ' 
program improvement and, depending on their 
nature, serve as a demonstration^to the e^uca- ' 
tional conimunity, the public, and policymakers 
that vocational education does irideed hnake a 
difference in preparing youth for jobs. 

Share procedures, instruments, and'.fesults With 
evaluation research'^unit and advisory council 
networks. Other dissemination dependent on 
findings.. ■ ^ T ' 

^ I . 

Since this outcome is addressed by VEDS, 
good opportunities will exist for comparing 
data-collection procerdures, responses, and 
interpretations of results. 



Source: 



R. L. Darcy^ Some Key Outcomes of Vocational Education: A fieport on Evaluation 
Criteria^ Standards, and Procedures. Co\umbus, OH: National Center for Research 
in Vocational Education, The Ohio State University^ 1980, pp. 42-43. 



.. \ - ■ ' - . CHAPTER V . " ' ■ - - . 

DESIGNING, PLANNING, AND CONDUCTING 

EMPLOYER FOLLOW-UP STUDIES , 

• . • • ■ " -/ ■-., » ■ ' ■ • -, ^ 

• • . : . ' \ / . , ' ' . \ ;• ^ . ' ^ 

This chapter identifies major decisions apd steps required in the design, planning, and qonduct I 
of an employer follow-up study^. This section is structured around questions and decisions the ". . • - 
e\/aluator must address. No attempt.has been made to cbver^in depth all asBects bf follovy-up studies : 
or the survey research and evaluation methodology. INIumferbus textb^^ providing a thorough \ 

treatment; therefore, selected references are made throughout the nari;atlve. Also, other publications 
in the National Center's Evaluation Handbook series hayjeralready address^^ those .important aspects 
at length and are referenced for the reader^s further information needs. ^^T^^^^ of this 

chapter are adapted from the publication^ Evaluation Handbook: Guidelines and Practices fdrTollow- 
£/p^Sf6/t//e^, by S. J. Franchakand J. E. Spirer 1978. /- 

Undertaking an assessment of employer satisfaction centers arounJ four phases: (1) preparation, 
(2) data collection, (3) data analysis and report preparation, (4) diSsemihation of results and feedback. 
Figure 3 presents 3 flowchart for conducting an assessment of employeri satisfaction of training and 
job performance of former vocational students, r - 1 



. Preparation 

/ 



The initial task requirdes an outline of a scope. of work and a detailed definition of tasks. Jime 
lines need to be established along with realistic target dates for completion. The assignment of staff 
requires a careful analysis of tasks to be completed, qualifications of persoris, and budgetary constraints. 
Persons responsible for the follow-up at the loctal educational agency (LEA) level typically function in 
''the roles of pfersonnersefvice/guidance and counseling, or in some^bases individual teachers fake on 
this responsibility. At the statue education agency (SEAi^ tev-^l, personnel located in the research, planning, 
or evaluation units generally are responsibly for the adm^nfe^ration and conduct of the assessment 
process. Table 2 presents a; useful guide'outOning the majoi decisions and questions the evaluator must 
address in conducting an employer follovv'-afp study. ^^^^^^ ' 

Developing Fbllow-up Study Objectives / 

- ■ ^ . --^ '■ I ■ ' • ■ • ^ i 

. Objectives must be clearly delineatecl because the assessment that addresses the wrong questions ; 
or doe's^fvot' provide a deq information is not successful. The primary purpose of this type of 
evaluation is to provide information for bcdgram improvement and accountability! Results are 
intended to influence the decision-mak^ process. The employer satisfaction results should answer 
several questions about vocational educ^on program quality: ' 

r ^ • ' . ■ ■ 

' 1 . Administrators and teachers want to know NA/hat vocational programs are accomplishing ' 
r and whether improvements are needed. ' ' .' 
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2. SEAs and federal agencies need to know whether programs are implemented as 
intended by the local and state plans. 

3. Administrator^nd teachers considering similar programs want to know what strategies , 
and methods work and why. 

Considerations of the potential users of the follow-up results will help in clarifying the specific ' 
objectives of the study. Before the final study goals and objectives are developed, the evaluator must 
determine if the training program and job performance of former students can be evaluated, and if 
It is feasible to obtain the required information. In other words, has the vocational program'addressed 
those job performance dimensions being assessed by the employer and does the employer have full 
knowledge of the training.curridulum? 



Defining the Data Collection Method 

Common data collection metliods for obtaining employer follow-up information include: C 
^ .1. Mail survey ' 
2. Telephone interview 
. 3. Personal interview 

■■■ . ■ 

The questionnaire used m each of these methods includes a list of questions or a request for a summary 
statement about the former vocational students as employees. Another approach requires asking the 
employer to respond.to a list of questions about an individual employee who has had vocational 
education training. 

The advantages and disadvantages of each methold jrjust be considered in light of costs, time, and 
personnel responsible for collecting the data, The persoj^f^njfer^^ study method) usually' 

provides the most useful and reliable, information bri^iSf^loearcomm unity or.jabor market area. But 
if the/sample of employers to be interviewed is large,'"possjbly a statewide area, the time and travel 
costs may be prohibitive^ An alternative would be a,,tel&pWrrg^r\Aey or a mail survey. The mailed 
survey is considered^most cost effective when the sample is large. However, this method requires 
many tasks to address the problems dealing with response rates. (These extra efforts could include 
the preparation and mailing of preaddressed and prestamped envelopes, and provide various monetary 
incentives.to return the instrument.) Dillman (1978) presents an excellent discussion on the advantages 
arid disadvantages ot the telephone and mail survey and also provides cost estimates relative^to the 
•use of both methods. - ^ ■■f.')' 

^ recent statewide effort to conduct an emj5loyer follow-up by; telephone provides val^ 
infornFiatibn relative p advantages and disadvantages of using this data collection method (CRW 
Associates 1980). The reader is encouraged to examine that report. , 



Ijdentification and Method for Sample Selection 

' Sample identification and samplirig method are critical elements in an employer follovv-up study. -' 
•Sampling, simply defined, is a technique that makes measurement of social phenomena possible. 
Sampiirig would not be necessary if the total population (e.g., ell employers defined for'the purposes 
of the study) participated in the study. Financial and time constraints generally prohibit population * 
surveys; ' . . . ; " 



A Michigan State Department of Education employer f0Jlow-up study provides an«excell/nt 
example for addressing the sampling problem dealing with emfeiioyer follow-up (The reader isf 
referred to the sampling plan developed by;S. G. Heeringa (19^9) for the Michigan Study ) Tfle 
sample was selected from employers of those students who indibated on the completer/leave/ 
follow-up (as defined by VEDS) that they were employed in a field related to their vocational 
training. ExcludedJrom the sample were those students who' wer^ self-employed or who wire in 
the military service. A stratified sampling design was used, resultink in the strkificatietlof popu- 
lations of students and employers by the completion status of studehtsand insttudtional program 
in which enrolled; .. ^ 

Generally, three questions must be addressed in arriving at a sampling method. ' 

First, .what procedures can be used to assure that the samples fairly represent 

the population to which the observations will be atrributed? Second, what 
can be done to assure that inevitable changes in the samples do not seriously 

affect their representativeness, and therefore, the quality of estimates of 

program effect? Finally, how large must the samples be to detect program 

impact with an adequate degree of certainty (General Accountinq Office 
1978, p. 15). 

- Problems of sample attrition and their effect' on 'representatives must be addressed. Changes in 
the composition of the samples can create differences that seriously bias the study. 

^ The determination of sample size is necessary to achieve\fhe required level of confidence and 
precision necessary in detecting effects. Heeringa (1979Movides a detailed discussion on establishing 
sampling rates for the employer follow-up with a discussion on levels of precision and sample size 
(p. 5-10). A number of textbooks on sampling are recommended for review and consultation These 
include: Socia/ Experimentation by H. W. Riecken and R. F. Boruch (New York: Academic Press 
1979); Survey Sampling bu L. Kish (New York: John Wiley, 1965); The Sample Survey by D P ' • 
Warwick and C. A. Lininger (New York:,McGraw-Hill, 1975); and Applied Sampling by S. Sudman 
(New York: Academic Press, 1976). 

Checklist 3 identified specific questions for helping to .develop an appropriate sampling strateav ' 
and method. . ' 

Identification of or Development of Follow-up Instrumi^r^t 

Designing or adapting an instrument for assessing employer satisfaction with training or former 
vocational education students' job performance requires answers to a number of questions. The 
decision to design an instrument or adapt one should be based on the consideration of certain factors. 
.These include such factors as study goals anci objectives, data collection methods, cost, and time, just 
to.mention the more obvious. 

If the decision is made to develop an instrument for a case study setting or mail-survey setting, 
adherence to the canons of instrument development are essential. Franchak ahd'Spirer (1978, pp. 
34-62) and McCaslin and Walker (1979, pp. 3-6) prpvide a systematic procedure fordeveloping a 
mail-survey questionnaire. Spirer (1979, pp. 40-60) provides a similar^procedure in developing a 
questjonnaire for use in a case study. Checklist 5 may be used to evaluate the employer follow-up 
instrument. „ ; 
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CHECKLISTS 
Sampling^Procedures 



''/.••. ■ ■ • • ■ 

Does your employer follow-up system's 

sarhple provide the following? 


Yes 


Somewhat 


No 


1. The desired information, accurately, at the 

rlp«;irprl nrirp? 


. □ 


□ 


■ □ 


naiiuuNi dctmpie seiecTion oyi 


i. 






a) simple random sampling? 

b) stratified sampling? 

c) cluster sampling? 

d) systematic selection? 

e) unequal probabilities of selection? 

f) '.rnultistage sampling? 


□ 




□ 


3. Nonrandom sample selection by: ^ 








a) haphazard sam'pling? 

b) judgment sampling? 

c) quota sampling? 


■ □ 
□ 


□ 


□ 


^ 4. An appropriate sample size statistically defined? 




□ . 


□ 


5. Response rate statisticaify identified? 


□ 




□ 
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CHECKLIST 4 

. Outline of Topics and Questions to be 
Considered for the Employer Questionnaire 



Importance ^ 



I. General Characteristics 

A. Questions pertaining to the respopdent 
. • business address . ' \ 

• title or position of respondent 
o tenure of position 

: B. Questions pertaining to the respondent's 
, business establishment 

• type of business 

• business function 

• location of business establish'fnent 

• number of employees 

\ • percentage of^dtal employees by sex, 
age, minorities; etc. 

• number qf new employees hired within 

• last twelve (12) months 

II. Young Worker Information 

A.. Questions pertaining to employment of 
young workers 

• employnient of individuals between 
16 anjd 19 years of age 

• employment of individuals between 
20 and 34 years of age 

• number pf new employees hired within 
. last.twelve (12) . montl^ between 16 

and 19 years of age I 

• number of n.ew.employees. hired within 
iat twelve (1 2) months between 20 
and 34 years of age 



Feasibility^ 



A * 5 = very high, 4 = moderately high, 3 = moderate, 2 = moderately low, and 1 = lower. 
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Checklist A, continued 



Importance Feasibility 

. • problems the employer encounters , 

with young'workers " 

• youTig workers' characteristics viewed . 

. positively by employer . / 

^' . B. Questions concerning employee training 

and educational requirements for 
* entry-level positions 

— : — . • minimum training and educational 

requirements for entry-level positions 

^ t level of initial skill training provided ■ 

to young workers 

— ~r- ; • source of training for young workers \, 

\ hired fn last twelve (12) months *' 

• ■ ' • " 

— — • employed individuals who have taken ' " ■ 

vocatipnal or technical education courses 

tr III. -Vocational Education Information 

A. Attitudes toward vocational education 

• degree of importance for public schools _ 

to provide opportunities to learn 
occupational'skills * 

: . • degree of emphasisithat public schools 

gi\)e to job training through vocational 
education programs y 

1- - • degree of importance for.public schools 

to provide students the opportunities 
to learn occupational skills. / 

. o degree of emphasis that public schools ' 

A give to job training through vocational . 

education ' 

— ^ — • importance of providing vocational 

education program^ to individuals under 
20 years of age ' 

• importance of providing vocational ' ■ 

education programs to individuals 

between 20 and 34 years of age - 
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Checklist 4, continued 



tmportahce 1 




FeasibiHty 


•* ; . ■ 


B. Collaboration between business 
establishment and school 






• ways the business establishment works 
with schools > 






^ • concerns the business establishment 
has in working with schools 


I •— ■ 




C\f Perceptions of the quality of vocational . 
education programs 






effectiveness of programs m preparing 
young people for job : 






\ • effectiveness of high school vocational 
programs in preparing people less than 
20 years of age 




e. 


effectiveness of community colleges and 
technical schools vocational programs 
in preparing people 20 years of age ^ • 
or older 

• relevancy of vocational programs 

• (secondary, postsecondary, and adult) 
to industry needs • 

• effectiveness (ccst-effectiveness) of 
providing occupational training 




v 


• recommendationsfor improving the 
delivery of vocational education 
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CHECKLIST 5 



Identifying Critical Factors in 
Developing the Employer Follow-Up Questionnaire 



Do the following statements apply to 
your follow-up questionnaire? 


Yes 


No 


Content ' " 






1. The first series of questions gain rapport with the respondent. 


.□ 




2. Ideas are sequenced according to the logic of the respondent. 






o. c;driy quebiions m a series are constructed so that the 
accuracy of subsequent responses can be checked. 


□ 




4. Questions that may be answered in a similar fashion are 
grouped together. 

w.. 

" Appearance 


□ 


□ 






5. The questionnaire is reproduced on high quality paper. 


□ 


□ 


6. Colored paper or colored ink is used to improve the 
questionnaire's visual appearance. ' 






7. The questionnaire is printed rather than mimeographed. 

■ ^ ■ ■ ■ ^ . ■ • .. • 
Format 


□ ■ 


□ 






o. 1 lie nL[e OT ine siuay IS Qispiayea near tne top o* the 
first-page. 


; □ 


□ -i 


9..- The name of the sponsoring agency is centered at the top 

nf thp firct nsrip 


□ 




10. Name, title,ind complete address of person to whom the 
form should be returned is included on the first page. 


□ 




11. Space is provided for the respondent's name and title. 


□ 




12. Questionnaires are pre-numbered. 


□ 


□ 


13. The purpose of the study is included \n an accompanying 
letter. . . 


□ 


□ 
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An important step in the designir g or adapting of an irjstrument is to pilot test it with a 
sample of employers. The purpose of the pilot test is to gain feedback about the draft instrument 
m terms of its clarity/ form, length, layout, and so forth. Erdos and Morgan (1970) provide a very 
succmct discussion on the pilot study. They state that pilot studies can be divided into several 
distmct categories by the purpose they serve. These include: 

.1 . Testing the quality of the mailing list 
2; Checking the percentages of returns 

3. Checking the effectiveness (in producing higher percentages of returns) 
of various segments of the data-gathering process, such as postcards, 
advance letters, incentives, and various types of follow-up efforts . ' ' 

4. Checking the occurrence of bias resulting from the wording of cards, 
letters, and questionnaires t 

.. 5. Checking on hovy well questions are understood and ansvvered ^ 

6. Checking the usefulness of information received 

7. Checking or even establishing a cost estimate (p. 84) 



Federal legislation such as the Confidentiafity and Priv^y Act must be taken into account in ' 
preparing to collect data on individuals. The primar> '-^qislatidn is the Family Education Rights and 
Privacy Act (Public Law 93-380, amended by Sectiori ^ of Public Law 93-568). Those responsible for 
the follow-up study must adhere to the requirements of the Act in the collection of the employer, 
satisfaction data. Each respondent in the study must be assured that information will be held in . 
confidence, and. that only aggregate data will be publicly disseminated for'the purposes of vocational 
education program improvement and accountability.' 

Thorough preparation for^employer follpw-uMata collection is essential. A vyell-defined data 
collection plan with specified quality control techniques is essential. No planning, poor planning, 
or planning not adhered to will r^esult in questionable, if hot misleading; results. The quality control 
techniques are needed to guarantee the integrity of the raw data because errors, omis.sions;^ or mis- 
' interpretations at that point will'bias.the statistical analysis (Gehera^l Accounting Office 19[78, p. 23). 
An example of essential quality control.techniquGs includeslpersonnel. training and recruitment to 
satisfy the gdals.and^objectives of the project. . 

The amount of training'necessary depends upon the initial ability of the staff and the complexity 
of the tasks. Another«technique includes pilot testing of data collection instruments, or interview 
guides if the case study method is used. Data editing is essential to ensure inadvertent errors do r)pt 
become a'part of the data base. Checks are eissential, so that all transcribed observations fall within 
defined boundaries. These include internal consistency checks for bpth quantitative and qualitative 
data. The reader is encouraged to revievy the California Statewide Longitudinal Study by Sheldon - 
and H&nter (1980) for comprehensive discussion of data editing problems and solutions. Finally, 
preliminary analyses can be performed during early stages of data collection. Early analyses can help 
to identify missing or incomplete jnformation, inconsistencies, or other discrepancies in the 'data. 
Based on these quality control techniques\ anticipated problems may be corrected to increase the 
probability of meaningful and useful data and information. . 



Procedures and Mechaijics for 
Colleeting Employer Follow-up Data 



\ 



Data Analysis Procedures 

^ The identification of this step could appear sooner in the flowchart but was positioned at this 
point because the employer follow-up instrument return rate mav play a critical role in the process. 
For example, if the mailed survey method fails to yield an acceptable response rate, the person in ' 
charge may have to revert to a contingency plan. Such plans could include reverting to alternative 
methods of collecting data (personal or telephone interviews) or weighing the data from those instru- 
ments which were returned. Another decision to make is whether to use automated or manual data 
analysis methods, or combination of the two. Franchak and Spirer (1976) present an analysis of 

these various data processing systems, including their advantages and disadvantages. 

< < , • ^. ' • • • 

The decision of whether to use atuomated, manual, or combination data processing will 
determine the analysis method.' Experience has shown that manual tabulation of data is satisfactory 
for up to 100 medium-length instruments. Once the number of employer folldw-up instruments 
exceeds 100, it becomes more efficient to use the speed and data manipulation abilities of the computer. 
Computer consultants or programmers are necessary at this stage. Experience proves it is useful to 
. , analyze the data several ways and to prepare several types of data displays for the various types of 
follow-up reports discussed in the next section. \ 



Writing Employer Follow-up Study Reports ; ^ 

Different audiences have, different information needs regarding the follow-up results. For example, 
states must report follow-up results to the federal government. Vocational school administrators, 
middle-level administrators, instructors,.and staff have an interest in the special needs program foliow-up 
study results. Advisory committee members, special educators, advocacy groups, and the general public 
also have needs regarding followHip study results. Franchak and Spirer (1 978) present a thoroTjgh 
discussion of follow-up data analysis and1-eporting procedures, identifying four different types of 
. follow-up reports: (1) highlight report, (2) executive sumnriary, (3) detailed^report, and (4) federal 
report. They also present a useful description of various graphic presentations, including.tables, bar 
graphs, line graphs, and histograms. Follow-up study concjusionsand recommendations should be . \ 

carefully thought out and clearsd through appropriate channels before they are included in the final 
report. Again, different audiences have needs for different types and levels of conclusions and recom- 
mendations. . . < 

^ • . ' - ■....•* ' ' V 

Publication and Printing Considerations • 

Because the final repprt represents the vocational school to many people, care should be taken 
to do a high-quality job on report printing. It may prove useful to explore creative approaches, such as / 
using the folder flyer, booklet, or brochure approach. The added printing expenses may be made up 
in improved good public relations for the vocational program. 

Developing an employer folloyv-up report evaluation form is an important step that is. often 
overlooked by evaluators. Just as follow-up study results should be used to revise and,;;improve the 
vocational special needs programs, so too should subsequent final follow-up reports be improved 
based on feedback provided in the report evjaluatiOn forms. These evaluation forms should be short 
■ and relatively simple. They could be includedin the printing of the final report as tear-out sheets, or 
\ they could be single sheets that may be folded and mailed back. ^ 
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Disseminating Follow-up Reports \ 

. Concerned audiences include but are not limited to: state-level needs personnel, project fundi 
officers, regional office of vocational bureau) vocational school administrators, advisory committe 
members, special education administrators, classroom/laboratory instructors, and other educators 
interested in the special needs program. A news release should be prepared for the news media with 
invitations to inspect the full final report. Stgte and national professional organization members may 
be interested in the report, and appropriate availability notices should be sent to their house organs 
and national magazines. A distribution list should be maintained for the file and for the information 
of the vocational school director. Last,' the report evaluation forms should be collected, analyzed and 
results tabulated for improving the next year's reports. ' . 
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APPENDIX 




ASSIESSMENT OF EMPLOYER SATISFACTION 
WITH TRAINING AND JOB PERFORMANCE 

■ ■ • . \ 



TO THE EMPLOYER: Please ask the person in your organization who has most direct responsibility 
for the hiring and supervising of personnel with less than a co.llege degree tp answer the questions 
below. This information is confidential; no data will be associated with the name of 'an individual or 
business establishment. , . . ' 



1. Address 

2. City , 



State 



3. Title or Position of Person Completing this Questionnaire: 

4. Howlorig in this position? (number of years) 



ZIP code 



5. Type of Business: 

□ Individual Proprietorship 

Partnership 

□ Cooperative Association 



□ Governmental, 
(please specify), 

□ Corporation 

□ Other 

(please specify). 



e. uHow would you describe the major activity of your business establishment? 



□ Agriculture 

□ Mining— Extractive 

□ Construction 

□ Durable Manufacturing 

□ Transportation/ - 
Communication Utilities 



□ Wholesale or Retail Trade 

□ Finance, Insurance, Real Estate 

□ Service 

□ Other 

(please specify) , 



7. Where is your plant or business establishment located? 

□ Rural 

□ Suburban 



□ Small City (less than 50,000 population) 

□ Urban (more than 50,000 population) \ 



8. What is the approximate average number of employees in your organiza 
. twelve (12) months? ^ 



tion during the last 



□ 1 - 9 

□ 10-49 



□ 50 - 59 ' 

□ 100-499 



□ 500 - 999 

□ 1,000 - 4,999 

9. Approximately what percentage of your total employees are: 

A. Teenage Workers (16-19 years of age) 
' B, Younger Adult Workers (20-30 years of :age)/ , J_ 

C. Minorities ' 

' D. Females . " i i ^ 

E. Union Members . ' 



□ 5,000 - 9,999 

□ 10,000 or more 



% 



10. Approximately how many new employees did your firm hire during the last.twelve (12) months? 

□ None ' 0 10-24 □ 50 - 99 □ 500 or more 

□ 1-9 □ 25 -49" □ 100-499 ^ 

1 1 . ,Does your business establishment employ individuals between the ages of 1 6 and 19? 

□ Yes ■ • ^ ; □ No / 

s. 

12. Does your business establishment employ individuals between the ages of 20 and 34? 

□ Yes ^ □ No . - 



If your business establishment does not employ individuals between the ages of 16 
and 34, you may stop here and return the questionnaire. ■ 

If your business does employ individuals between the ages of 16 and 34, please 
continue answering this questionnaire. " . 



13. Of the new employees hired in the lasl?twelve (12) months, how many were workers under the 
, age of 20? 

□ None^ □1-0-24 ^ □ 50 - 99 □ 500 or more 

□ 1-9 □ 25-49 □ 100-499 

14. Of the new employees,hired in the last twelve (12) montfis, how many were young workers 
between the ages of 20 and 34? 

□ None □ 10-24 ' □ 50 -99 " □ 500 or more 

□ 1-9 ^ 25-49 □ 100-499 • / 

15. Which of the following are serious problems with a substantial number of new erhployees in 
your business? " , 

□ Lack of basip ski I Is, "literacy, etc. 

□ Lack of job skills and knowledge 

□ Lack of acceptable work values, habits, attitudes, and motivation / 

□ Lack of job/work discipline 

□ Excessive tardiness and absenteeism . ► , 
. □ Failure to comply with company policies, procedures, etc. c . 

□ High turnover 

' □ Inflexibility . ^ 

□ Excessive use of alcohoj, drugs, etc. 

„ □ Poor personal relationships with other employees, . 

□ Low prodOctivi^. . ^ * . / 

□ inability to follow-through on assignments 

□ Abuse/theft of business property ^ ' " 

□ Inability to work without constant supervision 

□ Other (please specify) ^ . • ^ 



.16. yVhat are the minimum training and educational requirements for most entry-level positions 
in. your business? .v ' 

□ None ' * . / . . - . . 

□ High school diploma - 

□ GED or other high school equivalent 

□ Some skill training in vocational.pr technical education - • . ' 
^ □ Completion of a" course of study in vocational or technical education 

. '^D Demonstration of competency 

□ Competency certificate ; ' « j\ 

□ Previous work experience " . r^-r, 
. ^-Community college-or4echh'ical-school— - ^ ■ ^ ; 

□ Other (please specify), * ' ' ; 

1.7. How much and what kinds of initial skill training does your organization usually provide to ydiJng 
workers that do not have a baccalaureate degree? " ^ 

• Amount of Traintng (in days) - \ 



Kind of Training 



On-the-job training 
Vestibule 
Simulation 
Classroom 

Off-site training, Tuition-rebate ■ 
Off-site training, Tuitjon-subsidized 
Apprenticeship ' • 



' Less 
than 
1 


1-3 


4-5 


6-10 


11-15 


More .contm- 
than uous* 
15' ongoing 


□ 


□ 














□ 








□ . 






□ 


□ 








□ 








, □ 










□ 








□ 




□ 




□ 


□ . 




^ □ 





Other (please specify) 
18. Do 



Ma) 

. {by 

(c) 
^(d) 
(e) 
(f) 

(g) 

(h) 



more than 50 percent of your younger workers hired in the last twelve (12)' months have: 
* . ^ .. ' - . . ' Yes: . No 

Secondary vocational schoortraining?:> □ . □ 

Community college or technical school training? . □ □ 

Private school training? . . ^ □ □ 

College or university training? ^ ^ • □ \ cr 



Rrior work>xp^ietTce7e - \ 

Prior work experience not-related to the Job? 
Apprenticeship training? . ^ 

No special training? ' 



□ 
□ 
□ 



□ 
□ 
□ 



19. Has your business establishment employed individuals within the last twelve months who have" 
taken vocational or technical education programs? . 

□ Yes . □ No^ > ' 

If yes, please continue questionnaire. 

If no? you may stop here and return the questionnaire. 



'20. How important is it for the public sfehools to provide students less than 20 years of age, 
the opportunities to learn occupational skills? 

□ Very important - □ Not too. important □ Don't know 

□ Somewhat important' □ Not important at all J □ No opinion ' 

21 . How much emphasis do you think public schools should give to job training through vocational 
education programs (preparation of students for a specif ic occupation) for individuals less than 
20 years of age? t-. 

□ More emphasis > □ Less emphasis DDon'tknow 

□ Same emphasis □ No opinion 

22. How important is it^for the public schools to provide students age 20 and over the opportunities 
to l^arn occupational skills? , ^ 

□ Very important \ □ Not top important DDon'tknow • 

□ SorhewhatTmportant . .□ Not important at all □ No ppinion '* ... 

23. How much emphasis do you think the public schools should give to job traihing.through 
vocationaj.^educatidn prograr^ .. / 

□ More emphasis \ ;□ Less^ ^ - □Don't know ^ 
□/Same emphasis * / , ^ :□ No opinion » 

24. Please indicate the relative importance of eDch of the foMovying in providing vocational 
education through the public' schools for individuals £y/7fl^er J^^^ 



Addprod?ams; so that more stu'dents can enroll; 

Improve, update/and upgr^ the programs currently offered. 

I niprbve opportunities for the handicapped to receive , 

vocational education. - . .: 

j-t ' - . 

Improve opportunities for low-income students to receive 
Vocational education. ' - 

Improve opportunities for females ia vocationa.l education. 

Assure that indiyiduals-who don't go on to col lege have 
marketable skills. " ; * ., ry\ 

Provide work experience for vocational students, s. 

Supfiort^ feconomic development. .1 

Provide nontraditional (by sex) -training. 



Importance: 


Low 


High 


□ 


'□ 


□ 


□ 




■□ 




c 

□. 


□ 


' * □ ' 


□ 


□ 




. □ 




\ □, 







25. Please indicate the importance of providing vocational education programs to individuals 
• between 20 and 34 years of age. 

Importance: 



Low 


High 




□ 


□ 


Add programs, so that mpre students can enroll 


□ 


□ " 


Add programs, to provide training in more occupations. 


□ 


□ 


Improve, update, and upgrade the programs currently offered. 


n 

i_i 


n 

LJ 


improve opportunities tor tne nanaicappea to receive vocational 
education. \^'^^ 


□ 


□ 


iin|jiui/B up|jui luni Lies lor low-income stuoents to receive 
vocational education. 




□ 


^-Improve opportunities for females in vocational education. 


.□ ■ 


□ 


Assure that individuals who do not go on to college have 
rharketable skills.- 


□ ^' 


□ 


Insure that older individuals have access to training for entry or 
reentry into the job market, 


□ ■ 




Provide opportunities to learn at the business site. ^ 




-□ ■ 


* .Support economic development. " 


□ 




Provide nontraditional (by sex). training. - 




□ 


' Upgrade present workers with continuing education classes. 


□ 


□ 


' Provide work experience for vocational students. 



t 



26. In which of the following ways does your business work witK schools to support vocational 
education? (Pliease check all that apply.) .. 
High > Community 



Schools/ 
Vocational 
• Centers 


Colleges/ 
Technical, 
- Schools 




□ 




Suggesting new vocational courses 




□ 


Recommending what to teach in vocational courses 


□ 


in 


Recommending equipment and materialsJtQ_be used > 






' Providing equipment; and materials 






Designing facilities 


□ 


□ 


Providing facilities - 


-□ 


□ 


• Providing vocational studenWwith work experience 


□ 




Reporting employment status of vocational graduates 






Evaluating vocational graduates' job performance 


□ \ 


□ 


Serving on advisory committees 




□ - 


Providing teachers with work experience 


□ 


□ 


Recommending vocational teachers 






Releasing employees to teach vocational courses 


□ 




Providing training programs for the education community 


□ ; 


□ . 


Providing training for apprentices. 


□ 




\ J . Providing training for journeymen 


y 
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Very Fairly Not Too NotWetl Don't 
Good Well Well At All Know 



27. How well do the programs offered in 

your area prepare young people for jobs? , 

(a) Vocational education high schools 

(b) General education high schools ^ 

(c) Community colleges or technical schools 

(d) Proprietary schools 



-□ 


□ 


□ 


□ 


. ■ □ 


□ 


□ 


□ 


. □ 


□ 


□ 




□ 




□ 




□ 


□ 


□ 


□ 



28. How well do you feel the high school 
vocational programs offered in your area 
prepare people (less than 20 years of age) 
in each of the following? 



(a) 


Specific occupational skill 


. □ 


□ 


□ 


□ 


□ 


(b) 


Basic literacy skills ' 




□ . 




□ 


. □ 


(c) 


Problem solving 






□ 




□ 


(d) 


Decision making 




□ - 


□ 




□ 


(e) 


Self:directed learning skills 






□ 




□ 


(f) 


Acceptable work habits 




□ ■ 


□ 


□ . 


• □ 


(g) 


Acceptable work attitudes . ' 


□ 






■ ' □ 


□ 


(h) 


. Acceptable work values 




□ 


□ 




□ 


(!) 


Job application. skills .. 




' □ 




□ 


□ 


(i) 


Other (please specify:) 




□ ■ 


• □ 




□ 



29. How well do you feel the community colleges 
and technical schools vocatFonal programs 
offered in your area prepare people (ages 20 
' and over) in each of the following? 

(a) Specific occupational skill training □ 

(b) Basic literacy skills □ 

(c) Problem solving " □ 

(d) Decisionmaking □ 

(e) Self-directed learning skills □ 

(f) . Acceptable work habits □ 

(g) Acceptable work attitudes □ 
-::(!^hzAcceptable w^ — — : : z : □ : 

(i) Job application skills □ 
(j) Other (please specify:) . .* □ 



□ 
□ 
□ 
□ 
□ 
□ 

n 
□ 



□ 

□ 
□ 
□ 
□ 

□ - 

□ 



□ 
□ 
□ 
□ 
□ 
□ 
P 

□ 
□ 



□ 
□ 
□ 
□ 
□ 
□ 
□ 

□ 
□ 



50 



EKLC 



57 



Please check any of the following concerns.you have had in working with instructional and 
training agencies: 

Proprietary CETA ' ; . ' 

School Programs . " ;» 



High 
School 



Community College/ 
Technical School 



□ 
□ 



□ 



□ 



□ 
□ 

□ 



□ They are not interested in working 
with business and industry.- 

□ Conflicts or disagreements on goals, 

□ - Conflicts or disagreements over 

personalities. 

□ Conflicts or disagreements on policies 
pr regulations. 

□ Inadequate turnaround time to meet 
the need of business and industry. 



How^curreht are the vocational programs in your community? 
High Schools 



Completely 
Up-to-xiate 



Moderately 
Up-to-date 



Somewhat 
Out-of-date 



Entirely 
Out-of-date 



Don't 
Know 



(a) Course content ^ 




□ 






■ □ 


(b) Physical facilities 


□ • 




□ 


□ 


□ 


(c) Tools and equipment 


□ ■ 


□ 


□ 




■ P 


Community Colleges or Technical Schools 












(a) "Course content / 




□ 


. ■ ■ □ ' 


□ 




(b) Physical facilities 


□ 


' □ ■. 


□ 


□ ' 


^ □ 


(c) Tools and equipment 




■' □ 




□ - ■ 


□ 



In your local school district/ would you, as an individual; be willing to pay more taxes to support 
an increased emphasis on vocational education? 

□ Yes ; □ No ; ' 



33. What would be your main suggestion. on liow to improve vocational education programs in your 
community?^ .. . , " . ^ - . 
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GLOSSARY 



Some of the words used in the employer follow-up studies elude precise and clear definition. 
Those definitions that follow are intended to clarify the overall focus on the emjDioyer follow-uD 
studies, but' go beyond the definitions needed in doing this type of follow-up. Moreover, they are 
presented as a "definitational base" for which the comprehensive evaluation of vocation'areducation, 
■ of which employer fpllow-u'p studies are considered as an integral part. Tf^ey are not "carved in stone" 
and should not be interpreted so as to. foreclose alternative, potentially valuable definitions. 

A number of the definitions that follpw were. taken from the Federal Register, Vol. 42/'No.,191 
- Monday, October 3, 1977^ and pertiain to all sections of the Education Amendments of 1976, 
Title 1 1. Vocational Education; documentation pertaining to the National Center Educational Statistics' 
Vocational Educational Data System, and the National Occupational Informational Coordinating 
Committee's Glossary of Terms and Definitions Used in ah Occupational Information Program, J 978. 
K ... ■> ■ 

Attitude: A. relatively enduring ^system of affective evaluation reactions based upon and reflecting - 
the evaluative concepts or beliefs which have been learned about the characteristics of a social . 
objector class qf objects '(Shaw and Wright 1967, p. 10).. 

Belief: The emotional acceptance of a proposition or doctrine on what one considers to be adequate 
grounds (Eoglish and English 1958, p. 64). 

C ' : ■ ' ■ 

Civilian labor force; AH civilians sixteen years of age and over who are classified as either employed 
or unemployed. . ' <: . 

• - ' ' , * ■ . " " 

Completer .(of a VocationarEt(ucaticm Program): A student who finished a sequence of courses/ 
services, pr activities designed to meet an pccupatibnal objective v^ich purports to teach 
entry-level skills .{Section 104.404, Vocational Education Amend^nt Act, 1976). ' 

Cooperative. education: A combination program of vocational study and fDractice for persons through 
written Cooperative arrangements between school and employer. The'^program offers instruction, 
including required academic courses and related vocational preparation, by alternating study in ' 
school and supervised on-the-job training (Vocational Education Amendment Act, 1967; 
Handbook VI, modified). . 

Course: An instructional unit of area or field, or organized subject-matter and related learning 
experiences usually provided for the instruction. of students on a quarter/semesterjyear, or 
other prescribed length-of-time basis. It can.be offered for credit or noncredit (Handbook VI, 
modified). ^ 

Data: Things down or Sssamed; facts or figures from which calculations can be inferred; basic 
elements of information (Webster's New World Dictionary, 1967). 

■ ■ . ' V ■■' 
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Data base: A collection of data files which contain^ information usually related to:a ( ommon 
application. ' / ' 

Demographic information: Describes the population within a specified geographic area in terms of 
number, age, ethnic connposition, sex, work status, and/or other pertinent information. 

■ ' . V ■ ^ . 

Dual labor market: A dual labor market consists of primary and secondary occupations based on 
distinctions of race, sex, and age. Primary occupations are characterized by high job stability, 
• clearly defined career patterns, and a high degree of wojk involverh.ent. Secondary jobs are 
unstable and alienating. The dual theory cuts across traditional occupational categories and ' 
groups. 

Duration of unemployment: The period of time islapsed between a person's losing a job and obtaining 
another one. , 

Employed labor force: AM persons who do any work at all as paid employees in their area, profes- 
sional business, or firm; or who work fifteen hours or more as unpaid workers on a farm or in a 
business operated by a family member; and all persons who are not working but have jobs or / 
businesses frprh which they are temporarily absent whether or not they received pay for time off, 
or were lool<ing for other jobs (U.S. 'Census). 

Employee: A person hired by another or by a business, firm, etc. to work for wages or salary (Webster's 
New World Dictionary, 1967). ' - ' \ • 

Employer's.specificatiohs: Required skills, knowledges, aptitude, attitudes, training, or education, 
personal appearance, Snd job. prerequisites (such as license, certificate, union membership, etc.) 
demanded or desired o.f an applicant by employer. 

Employment Training: Training designed to enhance the employability of individuals by upgrading . 
basic skills through such courses as remedial education, work-orientation, English as a second 
language,=or training in the primary lanquage of persons of limited^'English usage; may be offered : 
aspart of institutional training (Public Law 93-203; Amen4ed). . 

Entrants ( Labor Force) : Persons who become part.of the labor force, may be new to the labor force, 
or may be reentering after a period spent outside the labor forqe. For a state or area, entrants 
may include migrants in the state area. 

Full-time employed: Persons working thirty-five hours or more per week (United States Census). 

Job: A position or erhploymenf^situation; work, either paid or unpaid. 

Job duties: The specific tasks the worker performs to accomplish the overall job purpose of an \ 
occupation (DOT modified). 

Labor force participation rates: The percent of the total population sixteen years of age or older \ 
which is actually in the labor force (i.e., meet the criteria of employed and unemployed). 

■ — ^ ■ -■ . ■ 

Labor market: The entire set of interlinked institutions and processes that determine the flow of 
the job opportunities and labor supply in both the short and long run. ' 
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Leaver (of a Vocational Education Program); A student who has been enrolled in and has attended 
a vocational education program and has jeft tHe^program without completing it; aisp includes 
those who leave the program voluntarily before completion, but leave with marketable skills, 
i.e., will be capable of obtaining and performing thejob for which preparation was directed ' 
■ '^(Vocational Education Act, modified Section 104.404). 



Local Education Agency (LEA): Any public authority^ legally constituted within a state to perform 
a service function for public elementary or secondary schools in a city, county, township, school 
district, or other political subdivision of a state (Section 160 c.2, P.L. 93'380);,any public V 
education institution or agency having administratil/^ control and direction of a vocational 
education program (Vocational Education Act, Section 108[9] ). 

Management information: Data, statistics, information used for free and objective decision processes 
and policy formation. \ 

Management Information System (MIS): An organized method of providing past, present,'and 
projected information relating to internal operations and external intelligence which supports 
the planning, control, and operational function of an orga'nization by providing decision miakers > 
with uniform information in the proper time frame (Kenneyan, Walter, "MIS Universe," Data 
' Management Vol. 8, September 1970, pp. 62-64). \ / 

' . ■ ■ ' - ' . . * 1 / ' ' 

Marketable skills: Competencies in a specific occupation or cluster of /elated occupations obtaine'd by 
.persons through training or other job preparation which Wet the hi^ring specifications of local 
employers. . ; .1 ' 1 



Nonresponse: In sample surveys, the failure to obtain information fron; designated individual for 
any reason (death, absence, refusal to reply). The proportion of such individuals of the sample . 
aimed at is called the nonresponse rate. It would bs better! however, to call this a "failure" rate 
or a "non-achievement" rate and to confine "nonresponse ' to those cases where tHe individual 
concerned is contracted but refuses to reply or is unable*to\do so for reasons such as deafness or 
illness(Kenda!landBuckland;M971, p. 105). ^ 

Occupation: The name or title of a job which identifies and specifies the various activities and 
functions to be performed. 1 

Occupational objective (Education): The expected outcome of training and other preparation as 
stated by an individual student. The objective usually is stated in terms of specific occupational 

. - ■ ' ■ 

Opinion: A belief that one holds to be without .emotional commitnient or desire, and to be open 
. td reevaluation since the evidence is not affirmed to be convincing /English and English 1958, 
• p. 358). . . 



Pilot survey: A survey, usually on a small scdie, carried out prior to the main survey, primarily to 
gain inforrnation td improve the efficiency of the main survey. For examplfs, it may be used to 
. test a questionnaire, to. a^(Jerfam time taken by a field procedure,! or to determine the most 
effective size of sampling bnit (Kendall and Buckland 1971, p. 114). 

Placeiment:.^ A po^itjbn-obt^^^^^ bY.j^.ps^son in an area of unsubsidized\employment either as a 
result of his/he'r^^pwri efforts^ftfef^ service or by referral to a^iob by the school or public 
^employment seryfce?V^ . ^f^'^^l^- . \ 
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Program: Program (or project) is defined to include instructional programs as well as legislative 
purpose programs. An instructional program is a planned sequence of courses, services, or 
.activitres designed tc meet an occupational objective. These programs are defined by the 0E\ 
.six-digit codes and include industrial arts program^and nongainf ul consumer and homemaking 
programs.^A legislative purpose^program is a course, service, or method of instruction which 
does not necessarily provide vocational skill training as a primary focus, but which formally is - 
organized by the state in response to legislative priorities (Policy Memorandum-BOAE DSVPO 
-FY9-2 Department of Health, Education and Welfare, Office of Education, Bureau of Occu- 
pational and Adult Education. Sent by: Charles H; Buzzell, Association Commissioner BOAE 

^ April 1979). ... > ^ - . 

Qualitative data: Detailed descriptions of situations, events, people, interactions, and observed 

behaviors; direct quotations from people about theiP experiences, attitudes,.beliefs, and thoughts 
and excerpts or entire passages from documents, correspondence, records, and case histories 
(Patton 1980, p. 22)., , 

. ■ ■' . •> 

Quantitative data: As contrasted with "qualitative'' data, should relate to data in the form of 
numerical quantities such as measurements or counts, it is sometimes, less exactly, used to 
describe material in which the variables concerned are quantities, e.g., sex, nationality, or 
commodity (Kendall and BucklandJ971, p. 121). ' ^ 

Sampling frame: The list, reasonable facsimile, of elements from which a probability sample is 
selected. Lists include organization lists, high sc>iools, industries, etc. (Babbie 1979, p. 175). 

Sampling unit: An element or set of elements considered for selection in some state of sampling. 
Examples of sampling units are census blocks, households, adults, etc. Primary sampling units, ^ 
secondary sampling units, and final sampling units would be used to designate the successive 
stages (Babbie 1979, p. 175). 

Specific Vocational Preparation (SVP): The amount of time required to learn the techniques, 
acquire information, and develop the facility needed for average performance in a specific 
job-related situation. This training may be required in a school, work, military, institution, or 
. . a vocational environment. It does not include the orientation training required by a fully ' 
qualified worker to become accustomed to thi^ special conditions of any new job. Specific . 
vocational.training includes. training given in any of the following circumstances:^ 

1. Vocational education 

2. Apprentice training / ' , . 

3. I n-plant training 

4. ^ On-the-job training 

5. Essential experience in other jobs 

Student (Vocational Education): An individual with a vocational objective who is enrolled in a 
vocational education program leading to entry or progress in a chosen occupational field. 

Value : Degree of worth ascribed to an object or activity (English and English 1958, p, 576). 
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_ ■ , . Employer Satisfaction, 

Baratta, M. K. Employer Eva/iiation of 1975 76 Occupational Graduates, Palos Hills, I L: Moraine \ 
Valley Community College, -1977. (ED 156 234) . • o ' 

. ■ ' 3. • 

Purposes of this employer iol low-up of the Moraine Valley Community College (MVCC) occupational 
program graduates were to determine (1) the degree to which graduates possessed the necessary 
.human relatiGr}s abilities, communication skills, problem-sblving skills, and technical and life skills 
to'f unction effectively in their positions; (2) graduates' work attitudes,performance quality and output 
quantity; (3) the relationship of MVCC achievement to employer evaluation; (4) strengths and weak- 
nesses in MVCC occupational program^areas, and (§),employers' willingness to hire additional MVCC 
graduates. ^ ■ . 

Data were collected by using the questionnaire mailed to 121 employers who hired 131 graduates 
. from business, health science, technology, and public service programs. Ernployers were asked to rate 
employees' various skills by checking the descriptors they considered most applicable: excellent, good, 
fair, poor, and not applicable. The rating questions were given under classified sections; (1 ) f^uman 
relations abilities-cooperating with fellow workers to get job done; (2) communication skills-organ\z\ng 
thoughts in writing; (3) technical s.^/7/sTrhandling equipment or instruments with speed; (A) problem- 
solving skills-hudgetmg time for carrying out the various work activities; and (5) life skills-adaptmg 
to new situations. 

Employers were also requested, using the same scale,* to assess.the overali attitude toward work;. the 
overall suitability for the kind of Job held; the ability to meet job. quality demand and the work-output 
quantity. Apart from this, assessment employers added comments such as: "This business graduate is ' 
3 very good worker, works at a steady pace, has very good organization, and gets along with other , 
workers/' and "I feel this technology employee lacks in organizational patterns, which should improve 
with experience in time to come." 

Findings were as follovys: (.1 ) Composite evaluation was consistent with the 1975 ratings. Employers 
. rated MVCC training as good. Public service graduates received the hiahest composite rating. (2) On 
human relations skills evaluation, public service graduates obtained more good-excellent ratings from 
their employers than graduates of other programs. (3) On communication skills'evaluation, the com- 
posite rate was Good. Health science graduates obtained more good-excellent ratings than graduates of 
other.programs, an'improvement from their 1975 ratings. (4) On technical-skills evaluation, the com- 
posite rate of four various occupational program graduates was good. Business program area graduates 
got good-excellent ratings, but this was the only program that received the same rating for all aspects 
of technical skills. (5) On problem-solving skills evaluation, the composite rating for'all graduates was 
good. Public service graduates had more good-excellent rating values than other program graduates, 
(6) On life-skills evaluation, the composite rate'jfor all program graduates was good-excellent. (7) On 
general-statement evaluation, MVCC occupational graduates vi^ere rated good-excellent with respect 
to quality of work, quantity of work, overall suitability, and overall attitude. (8) As to the relationship 
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of MVCQ achievement to employer's evaluation, it was found that favorable evaluations on human 
relations skills, problem-solving skills, life skills, quality of work, quantity of work, and a composite 
evaluation tended to be associated with graduates' good G.P.A.T.s. It was indicated that 85 percent of 
the employers would hire more MVCC graduates based on their present satisfactory experience with 
MVCC graduates. , ' 



Blackford, B.; Ruch, R.; Ahearn, J.; and Seymour, C. "Why Johnny Can Work: An Analysis of 
Employers' Ratings of Secondary Area Vocational-Technical Center Graduates." Pontiac, Ml: 
Northeast Oakland Vocational Educational Center, 1979. 

The stated purpose of the study was to obtain a measure of how vocational graduates compare with 
,pther entry-level employees in terms of the hiring considerations which employers feel are important 
and in terms of specific occupational skills and attitudes, A sampl^. was selected consisting of recent 
graduates from all program areas. Their employers vvere identified and interviews were hfeld. The 
employers were asked to rate the vocational, education graduates on hiring considerations, skill: prepara- 
tion, jobproductivity, basic skills, decision making, and other factors. Employers were also asked to 
compare these ratings with those they felt they wou[d give other entry-levej employees who were not 
vocational education graduates. Findings were that vocational education graduates consistently were " 
rated, higher than other entry-level employees in all areas, although graduates from some program areas 
were evaluated higher than others. ^ 



Feltsehausen, J. L. et al. "Fojlow-Up Report on Illinois 'Class of '71 ' Occupational Program Alumni, 
Final Report/^' Charleston, IL: Eastern Illinois University, Center for Educational Studies.:. 

The stated objectives of the study were to develop a follow-up data syste.m and to test the system by 
conducting an extensive follow-up study of the relationship between occupational programs and labor 
market experiences. Respondents included all 1971 occupational prograrn graduates from 2d percent 
of Illinois schools and their employers. Employisr satisfaction with training was made operational by 
employers rating how well the graduates were prepared in fifteen employment asp.ects. Over 73 percent 
of employers found the graduates to be well prepared in all aspects. There was a high correlation 
.between the graduates, job satisfaction, and employer ratings of preparedness. 

Gell, R. L, and Jones, R. F. The Employers II:. A Survey of Employers Who Have Hired Career 
Program Graduates of Montgomery Community College. Rockville, MD: Montgomery Cpllege, 
Office of Institutional REsearch and Analysis, 1975. (ED 1 12 997). - \ 

This study was a continuation of an annual follow-up study of employers. Purposes of the study,- 
mitiated in the fall of 1973, were (1):to identify employers' attitudes toward hiring Montgomery. 
Community College graduates, and (2) to involve employers in the evaluation process of college \ 
programs: \ 

Data were collected from seventy-seven employers who had hired career curriculum graduates of the. 
1974 class of the Montgomery Community College. Employers were surveyed to indicate whether ' 
they preferred to hire a prospective employee with an associate degree or a person without a degree \ 
but with some college education. They were also, asked to comment on the probability of promotion \ 
for a degree-holding graduate based on the five-point scale: 1 -never, 2-unlikely, 2sometimes, 
A— usually, and always. Evaluations of graduates' program training, as well as job preparation, was 
another focus of the study. Employers were asked: Based on your own experience in supervising a 
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Montgomery College graduate, please indicate how adequate the coMege preparation was in job skills, 
communication ability, overall work attitude, and so forth, using.the fiur-point scale: '[-inadequate, 
2— adequate, 2— more than adequate, and A— not observed. 

Findings vyere as follows: (1) the majority of the employers indicated tliat the associate degree was' ^ 
not essential for job entry, and they did not give preference to a person, who had an associate degree 
over a person without a degree; (2) employers considered earning an associate degree while on the job 
an enhancement to promotional chances in terms of job position, responsibilities, and tasks; and (3) 
the overall job preparation and performance were vievyed by employers as adequate to more than 
adequate; however, employers indicated that the significant weakness shared by most of the employees 
was their lafck of familiarity with the machines and equipment used in thp offices. 

An overall conclusion was that the findings of this study were virtually identical to those of a similar 
employer survey conducted in 1973. . . 1 

'' . ' ■ ! 

Gustafson, R. A., and Groves, P. M. How New Hampshire Employers Evajliate Their Employees- 
Implications for Vocational Education. Keene, NH: Keene State Coifege, 1977. (ED 149 030) 



Objectives of the research were (1 F to identify employers' expectations of employees' nonvocational 
skills, and (2) to determine, current employer practices in evaluating the prior and after-employment 
performance of personnel in New-Hampshire. 

Data were gathered by using personal interviews with twenty-three employers from businesses and 
industries of. different sizes and in varying geographic locations across the state of New Hampshire.^ 
Employers were interviewed to obtain responses to a standard set of prepared questions such, as: 
"Does your company/ organization use a written evaluation form which deals with the performance of 
your employees? If so, tould we have a sample cop'y:?!" and "When hiring new persongel, what six 
basic factors dp you consider most important?" In addition, employers were asked to rank the relative 
importance of ten work attitudes by using five criteria: extremely important, important, very important, ' 
not important, definitely not important, or does not apply. The ten factors included trainability, self- 
confidence, respectfulness, trustworthiness, optimism, working flexibility„responsibility, honesty, 
ambition, and cooperation. ' '' ' . -> 

Findings were as follo\A/s: (1 ) Graduates who know their career goals, who are confident of their 
qualifications, and who have cultivated human relations skills will impress the intervievver favorably 
and be more successfuhin obtaining a job. (2) In regard to the after-employment performance review, 
personal and attitudinal performances were considered as important as cognitive and psychomotor 
skills. Affective domain competencies appear to be more important when employees who hold salaried 
positioris are being evaluated. (3) Among the ten working habits, employers rated trustworthiness, 
and working flexibility highest in terms of importance while cooperation, respectfulness, and personal 
appearance were close seconds. Employers commented that an applicant should first try to impress 
the personnel receptionist, who will sometimes "code" the application form with brief general impressions 
of applicants and sometimes will decide either to pass applicants on to personnel qr to file the applica- 
tions. It was further commented that applicants should grasp the chance to convince^;the personnel 
manager during the general interview of their necessary qualifications. While being apprised of specific 
job skills by department supervisors, applicants should not only try to emphasize personal training 
skills but should also reflect an enthusiastic interest in the vyork and a willingness to work hard. 
Personality traits often play an important role in getting a job. . 

An overall conclusion>/as that if vocational program graduates are to be fully prepared for entering.and 
maintaining employment, a rhore substantial instructional effort in teaching desirable work attitudes 
and interpersonal relations is necessary. ^ i . 

■ ^ • ^ ; , ■ 
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Hamburger, M., and Wolfson, H. E. 1000 Employers Look at Occupational Education. Mew York: 
Board of Education of the City of New York, July 1969. 

This was the first in a series of studies planned to provide a sound base for curriculum redevelopment 
in. the occupational educationr programs of the New York City Board of Education. The objectives of 
the overall plan to make fundamental changes in occupational ed.uc^tion were as follows: (1) to 
improve the curriculum, (2) to extend occupational education to more youth, and (3) to introduce a 
multilevel approach with a variety of time and sequence organizations. 

; This study was designed to survey employers in an'effort to determine how to prepare a greater number 
and variety of young people for better jobs. Effective questions to be asked were discussed in workshops 
-one-composed of leaders from industry, commerce/labor; and education; the other composed of 
administrators, supervisors, and teachers in the New York City schools. 

^his project was conducted from February 1 to June 30, 1968, following a long planning period. From 
all the occupat/onal arenas in New York City, a sample of curricular areas was selected based on diversity 
importance, projected growth, and adaptability to a variety of school plants and organizations. The five' 
areas selected-business, health, automotive, metalworking, and electrical/electronics-included pre- 
dominantly male, predominantly female, and miked occupations. 

They include also a range of skill levels ... a balance between white- and blue-collar, between industrial 
and service, aad between stable and rapidly changing fields. Firms of, various sizes'were selected from 
' each category. Interviews were held with 1056 employers,Vith usable data obtained from 994. The 
final sample was not chosen through a rigorous sampling plan, but rather was based on the criteria stated 
above and on availability and cooperation. The authors note, therefore, that what was done was a case 
study of selected employers, rather than a study of a random sample. Interviews were held with presi- - 
dents, managers, personnel directors, and production supWvisors. ^ 

Project staff prepared an open-ended questionnaire and interview guide but, after using it, decided 
that a more objective approach would be more fruitful. The instrument was then redesigned. The 
redesigned questionnaire, which is provided in an appendix, basically follows a checklist format, 
although open-ended comments related to each question are sought as well. Included are questions 
•about job needs, job titles and duties, pay levels, and desired levels of education. The instnjments were - 
used during team visits TO each employment site. 

The data analyzed for this study consisted of responses to the survey questionnaires and impression- 
istic statements written by the interviewers. The interviewers also wrote recommendations for curriculum 
development. Quantitative data were. synthesized into questipn-by-question summary tables, and 
' qualitative data were content-analyzed so they could be condensed. Excerpts from comments and 
recommendations are presented, both in the text and in appendixes. 

Findings are presented primarily by occupational groupings in both a detailed chapter and a summary 
chapter. Among the findings discussed are these: 

• Contacts between schools and employers tend to be/'hit-or-miss." 

• Employers indicate that the kind of reading skills taught, such as reading work orders and 
° technical nnanuals, is as important as the amount taught. 

• Although a substantial minority of employers consider on-the-job training be?t for the 
most part employers feel that a school-job partnership is optimal for occupiitional training. . 

These and other findings are discussed in detail; many tables are presented; and limitations of the data 
are discussed as well as their implications. I \ 

i 60 



ERLC 



67 



Hanby, J.; Harper„R.; and Myers, L. A Comparison Study of the Benefits of Secondary and-^ * 
' Postsecondary Vocational Education. Portland, OR : Northwest Regional Educationaf * 
Laboratory, 1 978. . . , ^ . ' . 

The purpose of the study was to assess graduates' perceptions of their training, employers'. assessm,ent 
of graduates' training, and to conduct a comparison of perceptions of quality of life. The student ^ 
sample was selected from Montana -high school graduates who had been employed in Mohtana at least 
two years.,£mployers who often hire vocational education graduates were selected to-represent 
various sizes and types of businesses. Data were gathered through telephone surveys and mail question- 
naires. Employers were asked to rate 'postsecondary programs, secondary programs, and general 
academic programs as seen in employees wKo graduated from them. Employers were also asked to 
rate employee attitudes and to indicate whether or not they felt employees should have had'more^ 
e^cperience during training, Postsecondary programs were rated highest in terms of their ability to 
. prepare for work and also in terrns of their graduates' work attitudes. Most employers indicated a 
need for more job experience during training. 



Helmstedter, W. E. Community Employers' Evaluation of Bakersfield College Cooperative Work 
Experience Educav'on Program. {ED A53 673) . 

The purpose of the study was, to evaluate a work experience education program and to assess student 
iob performances as p6rcaived by present and past employers. Data on the current semester program 
(Fall 1979) were collected by using the training agreement evaluation form completed by 109 
employers randomly selected. A personal interview conducted by the program instructor/coordinator 
wil^Y employers enhanced. the completeness of data collection. Data were gathered from past program. 
employers through program evaluation questionnaires mailed to them. Employers and coordinators 
respectively rated student performance in seven major categories on a^fpur-poEnt scale: (1 ) limited 
progress; (2) needs further improvement; (3) njeets standards; and (4) exceeds standards. The seven 
categories evaluated Were job competence, dependabi^rty, attendance, initiative, appearance, relation- 
^ ships with peopje, and progress on the job. , ^ ^ 

Jhe employers were also asked to, answer such questions. as,' "How well does the student meet'"' . 
performance standards, organization, quality, and quantity of worki^nd "Do the sf .^dent's attitudes" 
reflect thoughtfulness, tact, respect, and.courtesy to supervisors, pelrs, subordinates, and dientele?" 

- ; ■ . / ' • ... • . ■ • ./ . y - ■ • ' . 

The five interviewing questions included the following: Do.you feel that the student/emplovee's. course 

- work at Bakersfield College has been rejevant to what he/she has been doinp on the job for ycL'? Has 
the.course work increased this person's worth as an empfoyee? On the basis of your experience with 
the student, wobid you^hire another v(^rk experience employee when another position opens? Do you 

^ feel the work experi^'^ program decreases employee turnover? and Does it decrease training costs 
^or the employer?^ndings were as fiillows: (1) Fall semester 1977.employers.were satisfied with the 
program students is to their job cortlipetence, dependability;, attendance, initiative, appearance, job 
progress, and relationship with coworkers. (2) Employers were generally satisfied with the Bakersfield 
College Work Experience Program. (3) Fifty-nine of the 109 employers felt that student/employee 

" college course work was relevan^^ while 52 indicated that the course work even increased 

student value as aq employee. Seventy-e'ight employers said they would hire other program students in 
the future. (4) In determining whether the program improved employee job satisfaction, vhirty-six 
considered it significantly effective, twisnty-three only moderately effectii^6, and ten, effective only 
to a limited extent. (5) Employers we/e somewhat divided in their Vesponses to training cost savings 
and employees. (6) About half of the employers felt that they contributed input to the program while 
the other half felt otherwise. An overall conclusion vyas that the majority of the employers had a 
favorable Impression of the program,, but they also identified areas that were weak and areas where 
communication needed to be improved. ^ ' 
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Iowa Department of Public Instruction; "A Report to the I ovya Department of Public Instruction 
on Employer Reactions to Employees Training in Preparatory Career Education Programs." 
Iowa Department of Public Instruction, 1977. - , ^ 

The stated purpose of the study was to determine whether employer evaluations of former students 
of preparatory career education programs in the technical higb. schools and community colleges of . 
Iowa could be collected and analyzed in a meaningful way. Data were collected from questionnaires 
administered to employers by experienced interviewers. Employers responded to items concerning 
, job skills, technical knowledge, quality and quantity of work, and attendance. Findings showed 
employers generally satisfied yvjth these^characteristics but concerned with employees' reluctance to 
accept responsibility and their heed for supervision, 

. Julian, R. E. "A Study of Employer Satisfaction with Graduates^of Oklahoma State University School 
of Technology.'' Ph.D. dissertation, Oklahoma State UjTiiiefsity . 

. The purpose of this study was to evaluate employer^' satisfaction with their employees in regard to 
their performance, cipnformance, dependability, personal adjustment, gnd general satisfactoriness. 
Data were collected from a questionnaire mailed to 30^ employees who were graduates from the 
^ f ou r- y ea r tec h n icafocc u pati onal irai n i ng-p rog^^^ 

from December 1973 through May 1975 and one mailed ta each of their employers. 

To determine whether employees saw their job satisfaction in the same relationship as their employers, 
the Minnesota Satisfactoriness Scale (MSS) was modified and' used by employees as a self-evaluation 
instrument on their job: satisfaction. Based the three criteria, net as weff, about the same, and better, 
employees were asked to compare themselves to others in their work group by answering such questions 
as: "Do you perform tasks i^equiring-variety and a change of methods?" "How good is the quality of 
ybUr;wbrk?'',''Do you perform repetitive ta^s?" Employers' were asked to rate the same twenty-eight : 
items as tfie employees did to evaluate employees' job satisfactoriness. 

. Findings included the following: (1 ) There wgs a tendency for employees' self-ratings to be slightly 
higher than those submitted by Jhe employers. (2) The correlation coefficient between the employers' 
and employees' performance rating vws the highest among the fivfcategories (i.e., performance, 
conformance, dependability, personaTadiustment, and general satisfactoriness). (3) The socioeconomic 
status, graduation date, and age of employees had veryslittle effect on general satisfactoriness rjatings : 
made by either employer or employee. Overall conclusions were that the employee group in this study 
fell within the satisfactory range on general satisfactoriness as rated by'the MSS, and that because of the 
ifSgreement in ratings, the data on job satisfactoriness could be gathered from either employees or 
employers. * ^ 

Kapes, J. J/AVTS Employer Follow-up Study. Bethlehem, PA: Lehigh University, 1977. (ED 051 576). 

Purposes of this study were to evaluate employers' satisfaction with the graduates' occupational com- 
petencies and to obtain the employers' general perceptions concerning the quality of Pennsylvania Area 
Vocational-Technical Schools' (AVTS) programs. Data were collected by usinq the mailed question- 
^ naires from stratified nonrandom sample of 1 ,000 Pennsylvania employers. The information' of the 
employers was provided regarding 3,285 AVTS students who had gradujted in 1976. Employers^were 
asked to provide general information^and to rate an eniplpyee's characteristics by using a 1 to 5 
Likert scale: (1 ) poor, (2)rbelow average, (2) average, (4) above average, and (5) excellent. Graduates 
were raf?^d^ their quality^of work; the degree to which they possessed basic reading, verbal, and 

i " ' ' ■ ' ' ■ ■ ' ■ ' 
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computational skills; the degree to which they possessed the general work nabits and attitudes^ 
necessary for success on the job, and so forth. By using the scales of most important (1) to least 
' important (7), employers were also instructed to rank the. importance of the job skills and employee 
qualities to their specif ic occupation. \ - 

In addition to this, employers were also requested to indicate their perceptions about the quality • 
of AVTS programs and graduatfes by cirpling.one of the five-point scales: (\) strongly disagree; 
(2) disagree, (3). neutral, (4) ggree, and (5) 5fro/7flr// aflrree.. Statements included: "I would rather hire 
a graduate from an AVTS than from a generator academic high school program/' "AVTS graduates 
are more adequately trained for specific job competencies than those from other secondary programs " 
and others. Thfe following were representative findings: (1) Based on the Likert scale, the graduates ' 
were rated highest on quality of work, basic skills, and near the middle on work habits and attitudes. 
(2) Employers ranked quality of work as most important and j6b attitudes second in importance. 

^ (3) Employers .would prefer to hire AVTS graduates a,nd.felt they possessed adequate specific job ' 
'< competencies; however, they did not agree that AVTS graduates displayed Letter work habits or needed 

,.less supervision. (4) In regard to communication skills, 46 percent of employers.would like more 
emphasis in this area. (5) The area of jot) attitudes and personal relation skills represented the single 
greatest concern to employers and perhaps the biggest challenge to schools. (6) Although employers in 

' Pennsylvania favored the present secondary vocational system. 



Overall observations revealed that (1) employers of the graduates responded favorably to their AVTS 
graduate employees; (2) employers of AVTS graduates differed in their evaluation in respect to the 
business they represented, and (3) regarding employers' perceptions toward AVTS programs in general, 
fmdmgs of this study should be useful to planners of vocational education in Pennsylvania. . 



' Kaplan, S. "Entry-Level Positions and Skiljs as They Relate tp the Local Schools' Programs as Perceived 
by Employers." Ph.D. dissertation. Temple University, 1975, Microfilm 75-28,283. ^ 

Purposes of this study were (1 ) to investigate the entry-level positions, available in the Philadelphia area, 
and (2) to deter mine whether the school training programs were adequate enough to prepare students 
: :o obtain entry-level jobs as they are perceived by employers. 

Data were gathered from tx free-response questionnaire-opinionnaire given to 202 of the largest 
business and industry employers in the Philadelphia metropolitan area. The five hundred or more 
people working for 50 employers represented one-third of. the local work force. Employers were asked * 
to indicate the generaLand specific skills or knowledge needed for employment, bu'sfness/industry 
^ satisfaction with the school product or preparatory trainings, entry-level position opportunity, and 
future labor market demand. Such questions ap the following were included: 'iPiease indicate the 
type and number of positions available for high school graduates in your firm for September, 1974." 
"Is there any way in which the vocational training programs_offered by the Philadelphia schools can 
be changed to better serve your needs? Please explain." "What newly created positions do you foresee 
for the fall of 1977?" and "Is the present vocational education in Philadelphia schools adequate for 
your piirpose?" ^ 

Findings vyere as follows-. (1) There were 160 specific entry-level positions available to high school 
graduates in 169 firms; among them, manufacturing business provicfed the majority of entry-level 
positions. (2) 60 percent of the entry-level, positions required on-the-job training or an apprenticeship 
progrartf in such fields as mechanics, electricity, service occupations, utility line installations, and so 
forth; (3) employers were not satisfied with certain nonskilled qualities of employees, such^as sloppy 
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appearance and speech, insufficient desire to work, failure to understand the handled product, 
weakness in following directions, lack of common sense, and desiring high pay for minimal qualifi- 
cations; i4) employers stated that some business training or background was needed to obtain and 
holdhold an entry-level position. (5) School should provide a training program more closely allied to 
industrial programs because better trained employees are needed. 

Overall conclusions were that employers placed considerable importance on training prospective 
workers in the areas of courtesy, and conversation, as well as in efficiency and neatness. Also, 
employers regarded the interpersonal area as very important including compatibility, pleasing 
personality, responsibility, honesty, and pride and enthusiasm in team work. Above all, employers 
expected high school graduates to possess a good background jn training that would help them 
obtain initial employment, achieve promotion, and advance on the employment arena. 

Kaufman, J. J., and Lewis, M. V. 'The High School Diploma: Credential for Employment?" U. Park, 
PA: Institute for Research on Human Resources, The Pennsylvania State University, 1972. - 

The stated objective of the study was to test the. hypothesis of credentialism where people are 
^evaluated with respect to the credentials they hold (in thisxase..the_high-schooi-diploma)vT-h^^^^^ 
was comprised of interested students from 1 ,200 dropouts who were randomly assigned to either a 
general education or a skifl training program. Employer satisfaction was indicated by employer ratings 
on work performance in the areas of occupational knowledge, manipulative skills, pe/sonal qualities, 
and overall preparation. Both groups were found to be very similar in employer ratings. 

Market Opinion Research. ''Employers and Youhg Adults Look at Vocational Education." Columbus: 
Ohio Advisory Couhcil'for Vocatipnal Education, 1977. ~ ' ' 

The purpose of the study was to determine the effects of vocational education on employabilityJobT^^ 
satisfaction, student satisfaction in training, and employer satisfaction in training. The sample consisted 
of.graduates of Ohio vocational education and other curriculum programs. Employees and employers 
were matched in pairs for interviews. Employer satisfaction was indicated by employer ratings of .. . 
workers on entry-leyei skills, technical knowledge, attitudes, supervisory potential, and school prepara- 
tion. Findings were that employers judged vocational education graduates higher f^-^n graduates of other 
programs. Employers also felt vocational I education graduates should have more t ,ning, especially in 
basic academic skills and responsibilities/' : 



Morton, J. B. Part-Time Adult and ^Employer Evaluation. Stillwater: Oklahoma State Department of 
Vocational and Technical Education. Division of Research, Planning and Evaluation 1977 
ED 173 545. _ _ - 

Purposes of this evaluation study were (1) to assess adult students' satisfaction .with the vocational 
training or retraining offered by the Oklahoma State Department of Vocational and Technicaf .. 
Education and (2) to deternrgne employers' satisfaction with adult students as to their job performance . 
and previous vocational training. Data were collected by using questionnaires mailed to 5,673 part-time ^ 
adult students who had enrolled in and completed the vocational preparatory training courses offered 
by the Oklahoma State Department of Vocational and Technical Education during .the fiscal year 1975. 
One hundred eighty-four employers who had hired those program completers also participated in the 
study. 
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Adult students were asked to provide their employers' names and addresses as well as their own *" 
personal, employment, and job information. They were also requested to evaluate program curriculum 
and equipment by checking the five-point scale: (^) excellent, (2) \/ery goodi {3) adequate, (4) made-, 
quate, (5) does not apply. Questions given, for example, were "How \A/ould you rate the skill training 
you received. in your vocational-technical program?" to asseii:. tha program instruction, the question 
"How would you rate the teaching quality of the instructor in your vocational training program- 
ID excellent, (2) good, or (3) poor," was asked. Employers were asked to evaluate employees' 
performance by chiscking any of the three blocks: above average, average, and below average. Areas 
to be measured included work quality and quantity, willingness to accept responsibility, cooperation 
with management, compliances with company policies, etc. The employers were also given five 
ratings: excellent, good, average, below average, and unsatisfactory for the overall suitability according 
to the employee. * . ^ ^ 

. • ■ ; ■ ' ■ - ■ ^ 

Findings were as follows; (1) More than seventy percent of the employee sample rat6d the skill, 
training as either very good or excellent; while only about 4 percent rated it poor. Fifty-eight percent 
said that the training program increased their ability to perform job duties, Seventy^one percent said 
.the training program'did not help them ^o increase skills in handling responsibilities assigned by the 
employer. (2) Ninety-three percent of the employees rated program instructors either good or excellent 
on questions concerning-teaching-qualityT^subjec^^ the extent to which the instructor 

was up-to-date in the field, andinterest in the student. Seventy-two percent of the employees rated 
the instructor's interest in their work progress after graduation excellent, while twenty-eight percent 
rated it poor. (3) About forty-five percent of the employers rated the aduit students' quality of work 
above average, while forty-hinepercent rated the quality as average. Almost 41 percent rated the 
quantity of work as above-average and 50 percent rated it as average. 

In respect to the overall suitability of the employees, 39 percent of the employer:S rated it excellent. 
(4) In regard to employer's satisfaction with the employee's vocational training, more than 90 percent 
were satisfied with training. Employee failure to meet employer expectations in \A/ork attitude was 
the major cause of employer dissatisfaction. 

OveraTlconcTijsions~were-that^ and their employers were very satisfied with part- 

time adult preparatory training jn Oklahoma, and~althl)u"gh~attention may-be^needed tojmprove work 
attitudes, the vocational training was considered generally adequate in that state. 7^ ~ ' — 



A Program Review of Secondary Vocational Education in Ohio: Job Placement and State Funding. 
Qhio Legislative Service Commission, Staff Report No. 126, April 1978. 

This study was done to provide feedback on the results of earlier vocational education policy and 
funding decisions and to provide useful information gnd guidelines for future legislative decisions. 
The study focuses on the job placement of vocational graduates (at the secondary level) and on related 
issues, such as graduate job satisfaction and graduate performance as evaluated by employers. The 
issue of how state funds can be better distributed to meet program costs is also addressed. 

Ten vocational education planning districts (VEPDs) were randomly selected from the sixty-five that 
had graduated their^first senior class by 1976. Four-day field visits were made to schools in these 
districts and one-day visits were made to Akron City Schools and to a Joint Vocational School in the 
Columbus area (on the. recommendation of the State yocational Education Division that they had 
excellent placement programs). One hundred sixty-six employers were interviewed by phone. Formal 
questionnaires were administered to vocational instructors and placement coordinators. The sample 
districts are described in an appendix/ . • 



The rest Its are discussed in great detail, including consideration of intervening factors, such as 
availability of employnfient. • 



The following findings are reported: 

• Vocational programs similar to those offered by technical colleges or propri^itary schools 
' do not have good placemen! records. 

• More than 90 percent of the graduates indicated that they would enroll in vocational 
education again. 

• Students and teachers expressed concern with technical skills, whereas employers stressed 
employability skills and adaptability. 

, • 58 percent of the employers hire vocatior:al graduates for jobs that\nonvocational high school 
■•: . graduates cannot fill without further training. . 

• 76 percent'df employers said they prefer to hire vocational graduates rather than general 
high school graduates. 

• A few students in each sample district said they would have dropped out of high school 
had there not been a vocational program. 

Two conclusions reached are that the vocational education program in Ohio has been relatively 
successful in'job placement, and that employers, while generally satisfied with the skills of vocational 
graduates, would favor increased communication with school officials. The need for a closer look at 
guidance and counseling and at the effects of vocational screening is suggested, and other areas for 
.further research are indicated. This study was well defined and thoroughly conducted. It is one of 
the few studies that compares perceptions of teachers, students, and employers. It is notable also for 
its discussion of economic conditions in the labor market and other variables that affect placement 
rates. Lacking%re (1) a discussion of the development of survey instruments, and (2) the collection 
df placement rate data for nonvocational graduates. 



Schowalte'r, Lynn.M. "The Relationship of High School Curriculum and Other In-School Character- 
isticts to Employment Success One Year After Graduation." Pennsylvania State University, 
. Department of. Vocational Education, 1 974. 

The purpose of the study'vi^~td~compare selected in-schooLcha^^^^^ to on-the-job success as 

^nrieasured by the employer's evaluation. Data were collected from empToyerTof the male-l^ — ! 

graduates of Altoona High School. Employers completed the Minnesota Satisfactoriness Scale, \ 
ranking their employees' performance on the job. Findings showed no significant difference between 
vocational and academic graduates as to employers' evaluation of satisfactoriness. 

Smiley, L . L. Employer Satisfaction with the Ski/is of Vocational Education Graduates in North 
Da/<ota. Grand Forks: Bureau of Educational Research and Services, University of North 
Dakota, August 1976. 

The initial data base consisted of all secondary and postsecondary sites in North Dakota with at least 
five vocational programs. Personnel in each program identified three major employers of program 
graduates and one or two self-employed graduates. The 482 names identified constituted the survey 
sample. Of -226 employers indicating a willingness to participate in a personal intervievv, 191 provided 
data. A questionnaire (appended to the document) was developed for use in the personal interview, 
which took approximately thirty minutes. 
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Twenty-one questions were asked, several with multiple parts. Some of the questions were in a 
rating-scale format, some in the form of checklists, and others were open-ended or required a yes 
or no response. They addressed issues of general satisfaction with employees; degree of specific 
knowledge, skills, and abilities employees possessed; and ways of improving training. 

Findings are presented in the order that questions appear on the interview schedule. Responses to 
each question are summarized in tabular form and discussed briefly. It is left to the reader to dravy 
co.nclusions. The reported findings include the following: 

• A speci^'ic employer comment is, "Most of the vocational education graduates come to work 
with skills well developed, but the training programs have failed to offer the .students anything 
that prepares them for mov'ng into supervisory or managerial positions." 

• Nearly 90 percent of the employers ranted the vocationally trained employee high to average 
in comparison with those without vocational training. Many could not make a comparison 
because certification requirements for entry-level job placement prevented a nonvocationally 
trained individual from being hired. 

Several needed areas of improvement for job applicants were liste^d. Heading the list (in orders 
of frequency of response ) were attitudes toward wo rk and appearance. Also listed were 
courses that vocationally trained students should have, with courses in communication, 
speaking effectively, and work orientation at the head of the list. Need for a course in success 
in marriage was listed by 88 percent of the respondents. 

• Many employers contact schools to recruit qualified employees. Some (including out-of-state) 
-schools are regularly contacted because such individual schools have gained reputations for 
quality, good skill development, and so forth. 

• A list of some of the general, comments employers made about vocational education programs 
and the graduates is provided. Among the comments are: "Realities of position, need to be 
made clear to the students," and "Secretaries trained asmedical secretaries are less capable 
than those with no specialized training." 

• Several employers expressed the opinion that high school gr^iduates are sold a bill of goods 
when told they have a saleable skill after their high school vocational training, whereas the 

■ intent of high school vocational training is to introduce young people to vocational areas and 
materials of those trades, not in any sense to provide a terminal degree. One employer said, 
"If this high school level training has helped a young person to clarify vocational interest which 
can be furthered by additional schooling, it has served an excellent purpose." 

South Carolina Advisory Council on Vocational and TechnicaLEjiucaiion/^^7^^^^ of Vocational 

and Technical Education, Columbia: South Carolina Advisory CounciTorrVocational and 

Technical Education, February 1976. (ED 126 315) 

The'South Carolina Advisory Council on Vocational and Technical Education culminates its 
evaluation^ responsibilities each yoar by>reparing and publishing an annual evaluation report. It 
became apparent, however, that "if one'is to adequately evaluate the 'effectiveness' of programs, 
what better perspective exists than that of the (Employers?'' This study was initiated to explore 
that often-neglected perspective, \ 

Many questions were asked: 

Are technical and vocational centers good sources of prospective, trained employees? 

• Do the centers meet the needs of business and'industry? 
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« Do they meet the needs of the students? 

• How do graduates of the centers compare with other employees? ' 

• What is^ the quality of this education? / 

Recommendations were sought as well. ' , j 

Ten thousand employers from large companies within the state constituted the population for the 
study; questionnaires were sent to each of them. Usable returns were received by the cutoff date from 
1,161 employers, a response ra te of 12 percent. Characteristics of the respondents are discjssed and 
presented in tables. A cross-section of the South Carolina business and industrial community/and of 
the geographic regions of the state was represented. The survey form was mailed in October 1975. 
The questionnaire does not ask when employers had experience with graduates of vocational or 
technical centers. \ ' . .. I 

Among the conclusions discussed are the following: I 

• Employers consider both vocational and technical\centers'a good source of trained en-iployees, 
generally rating those employees higher than others. ^ 

• Both vocational and technical centers meet the needs of business and industry and of the 
students, but they excel at meeting the needs of students. ^ 

• Employers are more familiar with technical education centers/colleges than with area 
vocational education centers. . 

• Vocational and technical programs are oriented more to-manufacturing than to other 
categories of business and industry. 



Sundi, R. L. "An Investigation of Selected Tasks That Affect Job Performance of Graduates as 
Perceived by Trade ancl Industrial Education teachers and Employers." Ph.D. dissertation. 
University of Cincinnati, 1971 ViVlicrofilm 72-9255. ^ 

Purposes of this study were (1 ) to investigate those tasks which affect the job performance of 
employees in the world of. trade and industry, and (2) to assist teachers in developing meaningful 
ir^structional programs that will meet industrial needs as well as technological changes. Data were 
collected by using the questionnaires mailed from 415 trade and industrial edjjcation program 
teachers from Greene Joint Vocational School and the Hamilton City^Schoo I District, as well as 
.from 132 employers who had hired students from trade and industrial education programs in the 
§tate of Ohio. , ' ^ ' • - - 

Both the employers and program teachers were asked to provide general background information and 
-to rate theJJOJob perfon^^ listed in the questionnaire, based on their !evel of 

importa-^ -e. The fivo-point \e\/e\s\N^e:( \)-not-important^^^^ (3) of average 

importa .ti, (4) of considerable importance, and (5) extremefy important. ThTl TO taskslnciuded 

the following: interpreting results of the work, handling several assignments, using the telephone, 
employing good English grammar, participating in group meetings, and cleaning own work area. 

Among the significant findings were: (1 ).of the total 1 10 tasks, both teachers and employers tended 
to>rate high in relation to job performance tho?e tasks which are attitude, interest, and motivation- 
oriented; (2) of. the total 1 10 tasks, 55 were considered important enough by both sample groups to 
be included in the curriculum.of a trade and industrial education program; (3) employers rated a 
total of 55 tasks (50 percent) at or above average in importance to job performance. 
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Overall conclusions were as fqilows. (1 ) There is ^ genern! agreement between trade and industry 
education teachers and employers as to those tasks which affect job performance. (2) Those tasks 
which fall in the category of interest, attitude, and motivation, such as assisting in the schedules With- * 
other departments and reviewing job problems with supervisor, contribute signif icantly toward 
graduates' job performance and should be included in the instructional program; (3) Those tasks 
which both groups rated /c 3 or above should be designed as technical information in the trade and 
industrial education program. » 

/ ■ 

. ■ ■ * / 

Talarzyk, VV. W. Percefjtions^ypcational Education in Ohio by Employers. Columbus: Educational 
Marketing Association, 1975. OYV43.2: P41 1/975, 

The purpose of this fesearc'n was to identify Ohio employers' attitudes and perceptions of the Ohio 
vocational educatio^ programs and its graduates. . . ^ %, 

Data were collectecf by mailing the validated question'-sire \o 1 ,900 randomly selected Ohio firms. 

Two hundred f ifty-bne employers completed and re^ 3d the que stionnaire, providing a 25 percent 

response rate. Empfoyers inc|ud'ed the users and non-users "of WcaTiona I education graduates. The 
vocationaTy trained graduates included those fronri agricultural education, business and office 
education, distributjve education, health education, heme economics education, technical education^, 
and trade and industrial ecjucation. The questionnaire contained attitude questions, attribute- 
imporrance questior^s, emplov/se-ratiog questions, open-ended questions, and classifications questions. . 
Employers Were askQd to indicate their satisfaction with employees by checking different-level 
answers in five blockWfor example: "Those having vocational education have more knowledge"- 
strong/y agree, agree/hndecided in opinion, disagree somewhat, or strongly disagree. "Responsibility 
and Ability to ioWof through-(1 ) most important, (2) secono most important ... (10) least 
important. Employers were also asked to measure the degree of graduates' competency in areas such 
as skills needed for job opening by marking one response-excellent, good, average, below average, 
or poor. In the section containing open-ended questions, employers were requested to respond to: 
"What do you see as the basic strengths of Ohio's yocationa! education^program?" arid 'What addi- 
tional information would be helpful to you in understanding and evaluating the state's vocational 
education programs?" Results were as follows: (1) 60 percent of the respondents agreed gr^rongly 
agreed th^t th,ey had had favorable experience with vdcaxionar education graduates, wRTlle 76 percent 
agreed or strongly agreed that vocational education students should have more practical, experiorice. 
(2) Dependabiiity was. considered by employers as the most important v^orkii.g trait and responsibility 
and ability to follow through were second most inripprtant. (3) VocationaKeducation graduates were 
rated strongest on skills needed for job opening, and willingness to learn new job skills^ or take training- 
courses. (4) Employers who did not hire vocational education graduates agreed more strongly than 
those who employed vocationally trained graduates in the following: Vocational education students 
are poorly trained. On-the-job training is better than vocational education. IMonusers of vocational 
graduates agreed more strongly than heavy users with graduates having more knowledge^ and most 
nevy employees today are well prepared for jobs.*(5) The importr.ice rating on employee attributes 
varied depending on the size of the firm, Lne percentage of employv"^es who were vpcational education 
graduates,;.and[the ty^^ employees hired. (6) Vocational education graduates were rated better 
than or equal to nonvocatibrially educated students on dl^^^^ bv medium (1-6 ^ 

percent) and heavy (over 6 percent) userj; of vocation \l education gra~dultesr(7) Firms with distribu- .""j 
tive education erriployees and those having health education employees tended to rate vocatio^nal 
education graduates higher as compared to nonvocationally trained gi-aduates on skilled needs for 
job o^penings, attitudes toward company/employer, and concern for productivity. A large majority , 
of all firms in the sample hired business and office, technical, trade'or industrial-educated employees, 
whije i 8w firms employed people in tfie other four categories of educational background. 
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Overall conclusions werL^ (1) that Ohio employers' attitudes toward and perceptions of vocational 
education were quite favorable; on almost all employee attributes, vocational education graduates 
were evaluated better than graduates of other curricula; (2) that the strengths of the program 
included the provision of concentrated training in one area, and graduates with more knowledge of 
their jobs than other new employees; whereas the program weaknesses were graduates lacking 
practical experience, programs lacking proper contact with employers, and too general a program; 

(3) that employers perceived the need for more practical experience for vocational education students; 

(4) that smaller firms tended to have a less favorable attitude toward vocational education than 
Jarger firms; and (5) that nonusers of vocational education graduates tended to have a less favorable 

image. of vocational education graduates than heavy users in the following: vocational education 
students are poorly trained, on-the-job training is better than.vocational education. Nonusers of 
vocational\graduates disagreed more strongly than heavy u^ers with the statements, graduates have ' 
more knowledge, and most new employe -s today are well prepared for jobs. 
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